Alex, I did a partial review - mostly server-side code. Please see my comments on GitHub. We will need more eyes on this, especially on Java client side.
Feature masks are a great addition! On Tue, Mar 31, 2020 at 1:05 PM Alex Plehanov <plehanov.a...@gmail.com> wrote: > Also, I've updated IEP regarding features masks (new backward compatibility > mechanism for thin clients) which was discussed earlier in this thread. > > вт, 31 мар. 2020 г. в 02:42, Alex Plehanov <plehanov.a...@gmail.com>: > > > Guys, > > > > I've finished IEP implementation for server-side and java thin client > side > > (ticket: [1], PR: [2]). Can you please review the patch? > > > > [1]: https://issues.apache.org/jira/browse/IGNITE-12835 > > [2]: https://github.com/apache/ignite/pull/7572 > > > > пт, 27 мар. 2020 г. в 19:11, Pavel Tupitsyn <ptupit...@apache.org>: > > > >> Agree with Igor, let's set deployment aside for now, it is out of scope > of > >> this IEP. > >> > >> > >> > >> On Fri, Mar 27, 2020 at 6:52 PM Igor Sapego <isap...@apache.org> wrote: > >> > >> > Hi guys, > >> > > >> > I like the proposal in general. > >> > > >> > Now for the task deployment - I think we should have separate API > >> > for it (and separate IEP I believe). Also, I'm not sure that this API > >> > should be a part of the API of any thin client as it seems weird to me > >> > to use Python client to deploy Java tasks. Control.sh or visor > proposal > >> > sounds much better. > >> > > >> > Best Regards, > >> > Igor > >> > > >> > > >> > On Thu, Mar 26, 2020 at 10:56 PM Denis Magda <dma...@apache.org> > wrote: > >> > > >> > > > > >> > > > Deployment API definitely needed as one of the next steps. > >> Currently, > >> > we > >> > > > are talking only about the first step (execution of already > deployed > >> > > > tasks). > >> > > > Also, I'm not sure about automatic redeploy and peer-class-loading > >> for > >> > > thin > >> > > > clients, I think it's better to have more control here and provide > >> API > >> > to > >> > > > explicitly deploy classes or jar files. WDYT? > >> > > > >> > > > >> > > Alex, agree that automatic redeployment is better suited for the > >> > management > >> > > APIs. How about adding this capability to our command-line tool > >> > > (control.sh, or visor cmd, or one new holistic tool). > >> > > > >> > > - > >> > > Denis > >> > > > >> > > > >> > > On Wed, Mar 25, 2020 at 1:04 PM Alex Plehanov < > >> plehanov.a...@gmail.com> > >> > > wrote: > >> > > > >> > > > Pavel, > >> > > > > >> > > > 1. Actually it can be solved on the client-side (and already > solved > >> in > >> > > PoC > >> > > > implementation). But I agreed it brings extra complexity for > >> > client-side > >> > > > implementation, will try to provide such guarantees on the > >> server-side. > >> > > > 2. ComputeTask has also "reduce" step which is executed on the > >> > initiator > >> > > > node. Binary-rest client implementation, for example, has such > >> affinity > >> > > > methods (to execute the task by name). I'm ok with removing it. At > >> > least > >> > > if > >> > > > someone will need it we can implement it again at any time in the > >> > future > >> > > > without protocol change. > >> > > > I've fixed IEP. > >> > > > > >> > > > Denis, > >> > > > > >> > > > Deployment API definitely needed as one of the next steps. > >> Currently, > >> > we > >> > > > are talking only about the first step (execution of already > deployed > >> > > > tasks). > >> > > > Also, I'm not sure about automatic redeploy and peer-class-loading > >> for > >> > > thin > >> > > > clients, I think it's better to have more control here and provide > >> API > >> > to > >> > > > explicitly deploy classes or jar files. WDYT? > >> > > > > >> > > > ср, 25 мар. 2020 г. в 21:17, Denis Magda <dma...@apache.org>: > >> > > > > >> > > > > Alex, thanks for preparing the outline. > >> > > > > > >> > > > > I'd like us to discuss an approach for compute tasks update with > >> no > >> > > > > downtimes on the servers' end. For instance, let's assume that a > >> > > > > Python/C++/Node.JS developer requested to update a compute task > he > >> > > called > >> > > > > from the app. Should we introduce some system level API to the > >> binary > >> > > > > protocol that can take a jar file (or class) and redeploy it > >> > > > automatically > >> > > > > with the usage of peer-class-loading? > >> > > > > > >> > > > > - > >> > > > > Denis > >> > > > > > >> > > > > > >> > > > > On Wed, Mar 25, 2020 at 5:47 AM Alex Plehanov < > >> > plehanov.a...@gmail.com > >> > > > > >> > > > > wrote: > >> > > > > > >> > > > > > Hello guys. > >> > > > > > > >> > > > > > I've implemented PoC and created IEP [1] for thin client > compute > >> > grid > >> > > > > > functionality. Please have a look. > >> > > > > > > >> > > > > > [1]: > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-42+Thin+client%3A+compute+support > >> > > > > > > >> > > > > > пт, 24 янв. 2020 г. в 16:56, Alex Plehanov < > >> > plehanov.a...@gmail.com > >> > > >: > >> > > > > > > >> > > > > > > We've discussed thin client compute protocol with Pavel > >> Tupitsyn > >> > > and > >> > > > > Igor > >> > > > > > > Sapego and come to the conclusion that approach with two-way > >> > > requests > >> > > > > > > should be used: client generates taskId and send a request > to > >> the > >> > > > > server > >> > > > > > to > >> > > > > > > execute a task. The server responds that the request has > been > >> > > > accepted. > >> > > > > > > After task has finished the server notifies the client > (send a > >> > > > request > >> > > > > > > without waiting for a response). The client can cancel the > >> task > >> > by > >> > > > > > sending > >> > > > > > > a corresponding request to the server. > >> > > > > > > > >> > > > > > > Also, a node list should be passed (optionally) with a > >> request to > >> > > > limit > >> > > > > > > nodes to execute the task. > >> > > > > > > > >> > > > > > > I will create IEP and file detailed protocol changes > shortly. > >> > > > > > > > >> > > > > > > вт, 21 янв. 2020 г. в 18:46, Alex Plehanov < > >> > > plehanov.a...@gmail.com > >> > > > >: > >> > > > > > > > >> > > > > > >> Igor, thanks for the reply. > >> > > > > > >> > >> > > > > > >> > Approach with taskId will require a lot of changes in > >> protocol > >> > > and > >> > > > > > thus > >> > > > > > >> more "heavy" for implementation > >> > > > > > >> Do you mean approach with server notifications mechanism? > >> Yes, > >> > it > >> > > > will > >> > > > > > >> require a lot of changes. But in most recent messages we've > >> > > > discussed > >> > > > > > with > >> > > > > > >> Pavel approach without server notifications mechanism. This > >> > > approach > >> > > > > > have > >> > > > > > >> the same complexity and performance as an approach with > >> > requestId. > >> > > > > > >> > >> > > > > > >> > But such clients as Python, Node.js, PHP, Go most > probably > >> > won't > >> > > > > have > >> > > > > > >> support for this API, at least for now. > >> > > > > > >> Without a server notifications mechanism, there will be no > >> > > breaking > >> > > > > > >> changes in the protocol, so client implementation can just > >> skip > >> > > this > >> > > > > > >> feature and protocol version and implement the next one. > >> > > > > > >> > >> > > > > > >> > Or never. > >> > > > > > >> I think it still useful to execute java compute tasks from > >> > > non-java > >> > > > > thin > >> > > > > > >> clients. Also, we can provide some out-of-the-box java > tasks, > >> > for > >> > > > > > example > >> > > > > > >> ExecutePythonScriptTask with python compute implementation, > >> > which > >> > > > can > >> > > > > > run > >> > > > > > >> python script on server node. > >> > > > > > >> > >> > > > > > >> > So, maybe it's a good time for us to change our backward > >> > > > > compatibility > >> > > > > > >> mechanism from protocol versioning to feature masks? > >> > > > > > >> I like the idea with feature masks, but it will force us to > >> > > support > >> > > > > both > >> > > > > > >> backward compatibility mechanisms, protocol versioning and > >> > feature > >> > > > > > masks. > >> > > > > > >> > >> > > > > > >> пн, 20 янв. 2020 г. в 20:34, Pavel Tupitsyn < > >> > ptupit...@apache.org > >> > > >: > >> > > > > > >> > >> > > > > > >>> Huge +1 from me for Feature Masks. > >> > > > > > >>> I think this should be our top priority for thin client > >> > protocol, > >> > > > > since > >> > > > > > >>> it > >> > > > > > >>> simplifies change management a lot. > >> > > > > > >>> > >> > > > > > >>> On Mon, Jan 20, 2020 at 8:21 PM Igor Sapego < > >> > isap...@apache.org> > >> > > > > > wrote: > >> > > > > > >>> > >> > > > > > >>> > Sorry for the late reply. > >> > > > > > >>> > > >> > > > > > >>> > Approach with taskId will require a lot of changes in > >> > protocol > >> > > > and > >> > > > > > thus > >> > > > > > >>> > more "heavy" for implementation, but it definitely looks > >> to > >> > me > >> > > > less > >> > > > > > >>> hacky > >> > > > > > >>> > than reqId-approach. Moreover, as was mentioned, server > >> > > > > notifications > >> > > > > > >>> > mechanism will be required in a future anyway with high > >> > > > > probability. > >> > > > > > So > >> > > > > > >>> > from this point of view I like taskId-approach. > >> > > > > > >>> > > >> > > > > > >>> > On the other hand, what we should also consider here is > >> > > > > performance. > >> > > > > > >>> > Speaking of latency, it looks like reqId will have > better > >> > > results > >> > > > > in > >> > > > > > >>> case > >> > > > > > >>> > of > >> > > > > > >>> > small and fast tasks. The only question here, if we want > >> to > >> > > > > optimize > >> > > > > > >>> thin > >> > > > > > >>> > clients for this case. > >> > > > > > >>> > > >> > > > > > >>> > Also, what are you talking about mostly involves clients > >> on > >> > > > > platforms > >> > > > > > >>> > that already have Compute API for thick clients. Let me > >> > mention > >> > > > one > >> > > > > > >>> > more point of view here and another concern here. > >> > > > > > >>> > > >> > > > > > >>> > The changes you propose are going to change protocol > >> version > >> > > for > >> > > > > > sure. > >> > > > > > >>> > In case with taskId approach and server notifications - > >> even > >> > > more > >> > > > > so. > >> > > > > > >>> > > >> > > > > > >>> > But such clients as Python, Node.js, PHP, Go most > probably > >> > > won't > >> > > > > have > >> > > > > > >>> > support for this API, at least for now. Or never. But > >> current > >> > > > > > >>> > backward-compatibility mechanism implies protocol > versions > >> > > where > >> > > > we > >> > > > > > >>> > imply that client that supports version 1.5 also > supports > >> all > >> > > the > >> > > > > > >>> features > >> > > > > > >>> > introduced in all the previous versions of the protocol. > >> > > > > > >>> > > >> > > > > > >>> > Thus implementing Compute API in any of the proposed > ways > >> > *may* > >> > > > > > >>> > force mentioned clients to support changes in protocol > >> which > >> > > they > >> > > > > not > >> > > > > > >>> > necessarily need in order to introduce new features in > the > >> > > > future. > >> > > > > > >>> > > >> > > > > > >>> > So, maybe it's a good time for us to change our backward > >> > > > > > compatibility > >> > > > > > >>> > mechanism from protocol versioning to feature masks? > >> > > > > > >>> > > >> > > > > > >>> > WDYT? > >> > > > > > >>> > > >> > > > > > >>> > Best Regards, > >> > > > > > >>> > Igor > >> > > > > > >>> > > >> > > > > > >>> > > >> > > > > > >>> > On Fri, Jan 17, 2020 at 9:37 AM Alex Plehanov < > >> > > > > > plehanov.a...@gmail.com > >> > > > > > >>> > > >> > > > > > >>> > wrote: > >> > > > > > >>> > > >> > > > > > >>> > > Looks like we didn't rich consensus here. > >> > > > > > >>> > > > >> > > > > > >>> > > Igor, as thin client maintainer, can you please share > >> your > >> > > > > opinion? > >> > > > > > >>> > > > >> > > > > > >>> > > Everyone else also welcome, please share your thoughts > >> > about > >> > > > > > options > >> > > > > > >>> to > >> > > > > > >>> > > implement operations for compute. > >> > > > > > >>> > > > >> > > > > > >>> > > > >> > > > > > >>> > > чт, 28 нояб. 2019 г. в 10:02, Alex Plehanov < > >> > > > > > plehanov.a...@gmail.com > >> > > > > > >>> >: > >> > > > > > >>> > > > >> > > > > > >>> > > > > Since all thin client operations are inherently > >> async, > >> > we > >> > > > > > should > >> > > > > > >>> be > >> > > > > > >>> > > able > >> > > > > > >>> > > > to cancel any of them > >> > > > > > >>> > > > It's illogical to have such ability. What should do > >> > cancel > >> > > > > > >>> operation of > >> > > > > > >>> > > > cancel operation? Moreover, sometimes it's > dangerous, > >> for > >> > > > > > example, > >> > > > > > >>> > create > >> > > > > > >>> > > > cache operation should never be canceled. There > >> should be > >> > > an > >> > > > > > >>> explicit > >> > > > > > >>> > set > >> > > > > > >>> > > > of processes that we can cancel: queries, > >> transactions, > >> > > > tasks, > >> > > > > > >>> > services. > >> > > > > > >>> > > > The lifecycle of services is more complex than the > >> > > lifecycle > >> > > > of > >> > > > > > >>> tasks. > >> > > > > > >>> > > With > >> > > > > > >>> > > > services, I suppose, we can't use request > >> cancelation, so > >> > > > tasks > >> > > > > > >>> will be > >> > > > > > >>> > > the > >> > > > > > >>> > > > only process with an exceptional pattern. > >> > > > > > >>> > > > > >> > > > > > >>> > > > > The request would be "execute task with specified > >> node > >> > > > > filter" > >> > > > > > - > >> > > > > > >>> > simple > >> > > > > > >>> > > > and efficient. > >> > > > > > >>> > > > It's not simple: every compute or service request > >> should > >> > > > > contain > >> > > > > > >>> > complex > >> > > > > > >>> > > > node filtering logic, which duplicates the same > logic > >> for > >> > > > > cluster > >> > > > > > >>> API. > >> > > > > > >>> > > > It's not efficient: for example, we can't implement > >> > > > > > forPredicate() > >> > > > > > >>> > > > filtering in this case. > >> > > > > > >>> > > > > >> > > > > > >>> > > > > >> > > > > > >>> > > > ср, 27 нояб. 2019 г. в 19:25, Pavel Tupitsyn < > >> > > > > > ptupit...@apache.org > >> > > > > > >>> >: > >> > > > > > >>> > > > > >> > > > > > >>> > > >> > The request is already processed (task is > >> started), > >> > we > >> > > > > can't > >> > > > > > >>> cancel > >> > > > > > >>> > > the > >> > > > > > >>> > > >> request > >> > > > > > >>> > > >> The request is not "start a task". It is "execute > >> task" > >> > > (and > >> > > > > get > >> > > > > > >>> > > result). > >> > > > > > >>> > > >> Same as "cache get" - you get a result in the end, > we > >> > > don't > >> > > > > > "start > >> > > > > > >>> > cache > >> > > > > > >>> > > >> get" then "end cache get". > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> Since all thin client operations are inherently > >> async, > >> > we > >> > > > > should > >> > > > > > >>> be > >> > > > > > >>> > able > >> > > > > > >>> > > >> to > >> > > > > > >>> > > >> cancel any of them > >> > > > > > >>> > > >> by sending another request with an id of prior > >> request > >> > to > >> > > be > >> > > > > > >>> > cancelled. > >> > > > > > >>> > > >> That's why I'm advocating for this approach - it > will > >> > work > >> > > > for > >> > > > > > >>> > anything, > >> > > > > > >>> > > >> no > >> > > > > > >>> > > >> special cases. > >> > > > > > >>> > > >> And it keeps "happy path" as simple as it is right > >> now. > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> Queries are different because we retrieve results > in > >> > > pages, > >> > > > we > >> > > > > > >>> can't > >> > > > > > >>> > do > >> > > > > > >>> > > >> them as one request. > >> > > > > > >>> > > >> Transactions are also different because client > >> controls > >> > > when > >> > > > > > they > >> > > > > > >>> > should > >> > > > > > >>> > > >> end. > >> > > > > > >>> > > >> There is no reason for task execution to be a > special > >> > case > >> > > > > like > >> > > > > > >>> > queries > >> > > > > > >>> > > or > >> > > > > > >>> > > >> transactions. > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> > we always need to send 2 requests to server to > >> > execute > >> > > > the > >> > > > > > task > >> > > > > > >>> > > >> Nope. We don't need to get nodes on client at all. > >> > > > > > >>> > > >> The request would be "execute task with specified > >> node > >> > > > > filter" - > >> > > > > > >>> > simple > >> > > > > > >>> > > >> and > >> > > > > > >>> > > >> efficient. > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> On Wed, Nov 27, 2019 at 4:31 PM Alex Plehanov < > >> > > > > > >>> > plehanov.a...@gmail.com> > >> > > > > > >>> > > >> wrote: > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> > > We do cancel a request to perform a task. We > may > >> > and > >> > > > > should > >> > > > > > >>> use > >> > > > > > >>> > > this > >> > > > > > >>> > > >> to > >> > > > > > >>> > > >> > cancel any other request in future. > >> > > > > > >>> > > >> > The request is already processed (task is > >> started), we > >> > > > can't > >> > > > > > >>> cancel > >> > > > > > >>> > > the > >> > > > > > >>> > > >> > request. As you mentioned before, we already do > >> almost > >> > > the > >> > > > > > same > >> > > > > > >>> for > >> > > > > > >>> > > >> queries > >> > > > > > >>> > > >> > (close the cursor, but not cancel the request to > >> run a > >> > > > > query), > >> > > > > > >>> it's > >> > > > > > >>> > > >> better > >> > > > > > >>> > > >> > to do such things in a common way. We have a > >> pattern: > >> > > > start > >> > > > > > some > >> > > > > > >>> > > process > >> > > > > > >>> > > >> > (query, transaction), get id of this process, end > >> > > process > >> > > > by > >> > > > > > >>> this > >> > > > > > >>> > id. > >> > > > > > >>> > > >> The > >> > > > > > >>> > > >> > "Execute task" process should match the same > >> pattern. > >> > In > >> > > > my > >> > > > > > >>> opinion, > >> > > > > > >>> > > >> > implementation with two-way requests is the best > >> > option > >> > > to > >> > > > > > match > >> > > > > > >>> > this > >> > > > > > >>> > > >> > pattern (we can even reuse OP_RESOURCE_CLOSE > >> operation > >> > > > type > >> > > > > in > >> > > > > > >>> this > >> > > > > > >>> > > >> case). > >> > > > > > >>> > > >> > Sometime in the future, we will need two-way > >> requests > >> > > for > >> > > > > some > >> > > > > > >>> other > >> > > > > > >>> > > >> > functionality (continuous queries, event > listening, > >> > > etc). > >> > > > > But > >> > > > > > >>> even > >> > > > > > >>> > > >> without > >> > > > > > >>> > > >> > two-way requests introducing some process id > (task > >> id > >> > in > >> > > > our > >> > > > > > >>> case) > >> > > > > > >>> > > will > >> > > > > > >>> > > >> be > >> > > > > > >>> > > >> > closer to existing pattern than canceling tasks > by > >> > > request > >> > > > > id. > >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > > So every new request will apply those filters > on > >> > > server > >> > > > > > side, > >> > > > > > >>> > using > >> > > > > > >>> > > >> the > >> > > > > > >>> > > >> > most recent set of nodes. > >> > > > > > >>> > > >> > In this case, we always need to send 2 requests > to > >> > > server > >> > > > to > >> > > > > > >>> execute > >> > > > > > >>> > > the > >> > > > > > >>> > > >> > task. First - to get nodes by the filter, second > - > >> to > >> > > > > actually > >> > > > > > >>> > execute > >> > > > > > >>> > > >> the > >> > > > > > >>> > > >> > task. It seems like overhead. The same will be > for > >> > > > services. > >> > > > > > >>> Cluster > >> > > > > > >>> > > >> group > >> > > > > > >>> > > >> > remains the same if the topology hasn't changed. > We > >> > can > >> > > > use > >> > > > > > this > >> > > > > > >>> > fact > >> > > > > > >>> > > >> and > >> > > > > > >>> > > >> > bind "execute task" request to topology. If > >> topology > >> > has > >> > > > > > >>> changed - > >> > > > > > >>> > get > >> > > > > > >>> > > >> > nodes for new topology and retry request. > >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > вт, 26 нояб. 2019 г. в 17:44, Pavel Tupitsyn < > >> > > > > > >>> ptupit...@apache.org > >> > > > > > >>> > >: > >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > > > After all, we don't cancel request > >> > > > > > >>> > > >> > > We do cancel a request to perform a task. We > may > >> and > >> > > > > should > >> > > > > > >>> use > >> > > > > > >>> > this > >> > > > > > >>> > > >> to > >> > > > > > >>> > > >> > > cancel any other request in future. > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > > Client uses some cluster group filtration > (for > >> > > example > >> > > > > > >>> > > forServers() > >> > > > > > >>> > > >> > > cluster group) > >> > > > > > >>> > > >> > > Please see above - Aleksandr Shapkin described > >> how > >> > we > >> > > > > store > >> > > > > > >>> > > >> > > filtered cluster groups on client. > >> > > > > > >>> > > >> > > We don't store node IDs, we store actual > >> filters. So > >> > > > every > >> > > > > > new > >> > > > > > >>> > > request > >> > > > > > >>> > > >> > will > >> > > > > > >>> > > >> > > apply those filters on server side, > >> > > > > > >>> > > >> > > using the most recent set of nodes. > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > var myGrp = > >> > cluster.forServers().forAttribute("foo"); > >> > > // > >> > > > > > This > >> > > > > > >>> does > >> > > > > > >>> > > not > >> > > > > > >>> > > >> > > issue any server requests, just builds an > object > >> > with > >> > > > > > filters > >> > > > > > >>> on > >> > > > > > >>> > > >> client > >> > > > > > >>> > > >> > > while (true) > myGrp.compute().executeTask("bar"); > >> // > >> > > > Every > >> > > > > > >>> request > >> > > > > > >>> > > >> > includes > >> > > > > > >>> > > >> > > filters, and filters are applied on the server > >> side > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > On Tue, Nov 26, 2019 at 1:42 PM Alex Plehanov < > >> > > > > > >>> > > >> plehanov.a...@gmail.com> > >> > > > > > >>> > > >> > > wrote: > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > > > Anyway, my point stands. > >> > > > > > >>> > > >> > > > I can't agree. Why you don't want to use task > >> id > >> > for > >> > > > > this? > >> > > > > > >>> After > >> > > > > > >>> > > >> all, > >> > > > > > >>> > > >> > we > >> > > > > > >>> > > >> > > > don't cancel request (request is already > >> > processed), > >> > > > we > >> > > > > > >>> cancel > >> > > > > > >>> > the > >> > > > > > >>> > > >> > task. > >> > > > > > >>> > > >> > > So > >> > > > > > >>> > > >> > > > it's more convenient to use task id here. > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > > Can you please provide equivalent use case > >> with > >> > > > > existing > >> > > > > > >>> > "thick" > >> > > > > > >>> > > >> > > client? > >> > > > > > >>> > > >> > > > For example: > >> > > > > > >>> > > >> > > > Cluster consists of one server node. > >> > > > > > >>> > > >> > > > Client uses some cluster group filtration > (for > >> > > example > >> > > > > > >>> > > forServers() > >> > > > > > >>> > > >> > > cluster > >> > > > > > >>> > > >> > > > group). > >> > > > > > >>> > > >> > > > Client starts to send periodically (for > >> example 1 > >> > > per > >> > > > > > >>> minute) > >> > > > > > >>> > > >> long-term > >> > > > > > >>> > > >> > > > (for example 1 hour long) tasks to the > cluster. > >> > > > > > >>> > > >> > > > Meanwhile, several server nodes joined the > >> > cluster. > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > In case of thick client: All server nodes > will > >> be > >> > > > used, > >> > > > > > >>> tasks > >> > > > > > >>> > will > >> > > > > > >>> > > >> be > >> > > > > > >>> > > >> > > load > >> > > > > > >>> > > >> > > > balanced. > >> > > > > > >>> > > >> > > > In case of thin client: Only one server node > >> will > >> > be > >> > > > > used, > >> > > > > > >>> > client > >> > > > > > >>> > > >> will > >> > > > > > >>> > > >> > > > detect topology change after an hour. > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > вт, 26 нояб. 2019 г. в 11:50, Pavel Tupitsyn > < > >> > > > > > >>> > > ptupit...@apache.org > >> > > > > > >>> > > >> >: > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > > > I can't see any usage of request id in > >> query > >> > > > > cursors > >> > > > > > >>> > > >> > > > > You are right, cursor id is a separate > thing. > >> > > > > > >>> > > >> > > > > Anyway, my point stands. > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > > client sends long term tasks to nodes and > >> > wants > >> > > to > >> > > > > do > >> > > > > > it > >> > > > > > >>> > with > >> > > > > > >>> > > >> load > >> > > > > > >>> > > >> > > > > balancing > >> > > > > > >>> > > >> > > > > I still don't get it. Can you please > provide > >> > > > > equivalent > >> > > > > > >>> use > >> > > > > > >>> > case > >> > > > > > >>> > > >> with > >> > > > > > >>> > > >> > > > > existing "thick" client? > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > On Mon, Nov 25, 2019 at 11:59 PM Alex > >> Plehanov < > >> > > > > > >>> > > >> > > plehanov.a...@gmail.com> > >> > > > > > >>> > > >> > > > > wrote: > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > > > And it is fine to use request ID to > >> identify > >> > > > > compute > >> > > > > > >>> tasks > >> > > > > > >>> > > >> (as we > >> > > > > > >>> > > >> > > do > >> > > > > > >>> > > >> > > > > with > >> > > > > > >>> > > >> > > > > > query cursors). > >> > > > > > >>> > > >> > > > > > I can't see any usage of request id in > >> query > >> > > > > cursors. > >> > > > > > We > >> > > > > > >>> > send > >> > > > > > >>> > > >> query > >> > > > > > >>> > > >> > > > > request > >> > > > > > >>> > > >> > > > > > and get cursor id in response. After > that, > >> we > >> > > only > >> > > > > use > >> > > > > > >>> > cursor > >> > > > > > >>> > > id > >> > > > > > >>> > > >> > (to > >> > > > > > >>> > > >> > > > get > >> > > > > > >>> > > >> > > > > > next pages and to close the resource). > Did > >> I > >> > > miss > >> > > > > > >>> something? > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > Looks like I'm missing something - how > is > >> > > > topology > >> > > > > > >>> change > >> > > > > > >>> > > >> > relevant > >> > > > > > >>> > > >> > > to > >> > > > > > >>> > > >> > > > > > executing compute tasks from client? > >> > > > > > >>> > > >> > > > > > It's not relevant directly. But there are > >> some > >> > > > cases > >> > > > > > >>> where > >> > > > > > >>> > it > >> > > > > > >>> > > >> will > >> > > > > > >>> > > >> > be > >> > > > > > >>> > > >> > > > > > helpful. For example, if client sends > long > >> > term > >> > > > > tasks > >> > > > > > to > >> > > > > > >>> > nodes > >> > > > > > >>> > > >> and > >> > > > > > >>> > > >> > > > wants > >> > > > > > >>> > > >> > > > > to > >> > > > > > >>> > > >> > > > > > do it with load balancing it will detect > >> > > topology > >> > > > > > change > >> > > > > > >>> > only > >> > > > > > >>> > > >> after > >> > > > > > >>> > > >> > > > some > >> > > > > > >>> > > >> > > > > > time in the future with the first > >> response, so > >> > > > load > >> > > > > > >>> > balancing > >> > > > > > >>> > > >> will > >> > > > > > >>> > > >> > no > >> > > > > > >>> > > >> > > > > work. > >> > > > > > >>> > > >> > > > > > Perhaps we can add optional "topology > >> version" > >> > > > field > >> > > > > > to > >> > > > > > >>> the > >> > > > > > >>> > > >> > > > > > OP_COMPUTE_EXECUTE_TASK request to solve > >> this > >> > > > > problem. > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > пн, 25 нояб. 2019 г. в 22:42, Pavel > >> Tupitsyn < > >> > > > > > >>> > > >> ptupit...@apache.org > >> > > > > > >>> > > >> > >: > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > Alex, > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > > we will mix entities from different > >> layers > >> > > > > > >>> (transport > >> > > > > > >>> > > layer > >> > > > > > >>> > > >> and > >> > > > > > >>> > > >> > > > > request > >> > > > > > >>> > > >> > > > > > > body) > >> > > > > > >>> > > >> > > > > > > I would not call our message header > >> (which > >> > > > > includes > >> > > > > > >>> the > >> > > > > > >>> > id) > >> > > > > > >>> > > >> > > > "transport > >> > > > > > >>> > > >> > > > > > > layer". > >> > > > > > >>> > > >> > > > > > > TCP is our transport layer. And it is > >> fine > >> > to > >> > > > use > >> > > > > > >>> request > >> > > > > > >>> > ID > >> > > > > > >>> > > >> to > >> > > > > > >>> > > >> > > > > identify > >> > > > > > >>> > > >> > > > > > > compute tasks (as we do with query > >> cursors). > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > > we still can't be sure that the task > is > >> > > > > > successfully > >> > > > > > >>> > > started > >> > > > > > >>> > > >> > on a > >> > > > > > >>> > > >> > > > > > server > >> > > > > > >>> > > >> > > > > > > The request to start the task will fail > >> and > >> > > > we'll > >> > > > > > get > >> > > > > > >>> a > >> > > > > > >>> > > >> response > >> > > > > > >>> > > >> > > > > > indicating > >> > > > > > >>> > > >> > > > > > > that right away > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > > we won't ever know about topology > >> change > >> > > > > > >>> > > >> > > > > > > Looks like I'm missing something - how > is > >> > > > topology > >> > > > > > >>> change > >> > > > > > >>> > > >> > relevant > >> > > > > > >>> > > >> > > to > >> > > > > > >>> > > >> > > > > > > executing compute tasks from client? > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > On Mon, Nov 25, 2019 at 10:17 PM Alex > >> > > Plehanov < > >> > > > > > >>> > > >> > > > > plehanov.a...@gmail.com> > >> > > > > > >>> > > >> > > > > > > wrote: > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > > Pavel, in this case, we will mix > >> entities > >> > > from > >> > > > > > >>> different > >> > > > > > >>> > > >> layers > >> > > > > > >>> > > >> > > > > > > (transport > >> > > > > > >>> > > >> > > > > > > > layer and request body), it's not > very > >> > good. > >> > > > The > >> > > > > > >>> same > >> > > > > > >>> > > >> behavior > >> > > > > > >>> > > >> > we > >> > > > > > >>> > > >> > > > can > >> > > > > > >>> > > >> > > > > > > > achieve with generated on client-side > >> task > >> > > id, > >> > > > > but > >> > > > > > >>> there > >> > > > > > >>> > > >> will > >> > > > > > >>> > > >> > be > >> > > > > > >>> > > >> > > no > >> > > > > > >>> > > >> > > > > > > > inter-layer data intersection and I > >> think > >> > it > >> > > > > will > >> > > > > > be > >> > > > > > >>> > > easier > >> > > > > > >>> > > >> to > >> > > > > > >>> > > >> > > > > > implement > >> > > > > > >>> > > >> > > > > > > on > >> > > > > > >>> > > >> > > > > > > > both client and server-side. But we > >> still > >> > > > can't > >> > > > > be > >> > > > > > >>> sure > >> > > > > > >>> > > that > >> > > > > > >>> > > >> > the > >> > > > > > >>> > > >> > > > task > >> > > > > > >>> > > >> > > > > > is > >> > > > > > >>> > > >> > > > > > > > successfully started on a server. We > >> won't > >> > > > ever > >> > > > > > know > >> > > > > > >>> > about > >> > > > > > >>> > > >> > > topology > >> > > > > > >>> > > >> > > > > > > change, > >> > > > > > >>> > > >> > > > > > > > because topology changed flag will be > >> sent > >> > > > from > >> > > > > > >>> server > >> > > > > > >>> > to > >> > > > > > >>> > > >> > client > >> > > > > > >>> > > >> > > > only > >> > > > > > >>> > > >> > > > > > > with > >> > > > > > >>> > > >> > > > > > > > a response when the task will be > >> > completed. > >> > > > Are > >> > > > > we > >> > > > > > >>> > accept > >> > > > > > >>> > > >> that? > >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > пн, 25 нояб. 2019 г. в 19:07, Pavel > >> > > Tupitsyn < > >> > > > > > >>> > > >> > > ptupit...@apache.org > >> > > > > > >>> > > >> > > > >: > >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > Alex, > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > I have a simpler idea. We already > do > >> > > request > >> > > > > id > >> > > > > > >>> > handling > >> > > > > > >>> > > >> in > >> > > > > > >>> > > >> > the > >> > > > > > >>> > > >> > > > > > > protocol, > >> > > > > > >>> > > >> > > > > > > > > so: > >> > > > > > >>> > > >> > > > > > > > > - Client sends a normal request to > >> > execute > >> > > > > > compute > >> > > > > > >>> > task. > >> > > > > > >>> > > >> > > Request > >> > > > > > >>> > > >> > > > ID > >> > > > > > >>> > > >> > > > > > is > >> > > > > > >>> > > >> > > > > > > > > generated as usual. > >> > > > > > >>> > > >> > > > > > > > > - As soon as task is completed, a > >> > response > >> > > > is > >> > > > > > >>> > received. > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > As for cancellation - client can > >> send a > >> > > new > >> > > > > > >>> request > >> > > > > > >>> > > (with > >> > > > > > >>> > > >> new > >> > > > > > >>> > > >> > > > > request > >> > > > > > >>> > > >> > > > > > > ID) > >> > > > > > >>> > > >> > > > > > > > > and (in the body) pass the request > ID > >> > from > >> > > > > above > >> > > > > > >>> > > >> > > > > > > > > as a task identifier. As a result, > >> there > >> > > are > >> > > > > two > >> > > > > > >>> > > >> responses: > >> > > > > > >>> > > >> > > > > > > > > - Cancellation response > >> > > > > > >>> > > >> > > > > > > > > - Task response (with proper > >> cancelled > >> > > > status) > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > That's it, no need to modify the > >> core of > >> > > the > >> > > > > > >>> protocol. > >> > > > > > >>> > > One > >> > > > > > >>> > > >> > > > request > >> > > > > > >>> > > >> > > > > - > >> > > > > > >>> > > >> > > > > > > one > >> > > > > > >>> > > >> > > > > > > > > response. > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > On Mon, Nov 25, 2019 at 6:20 PM > Alex > >> > > > Plehanov > >> > > > > < > >> > > > > > >>> > > >> > > > > > plehanov.a...@gmail.com > >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > wrote: > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Pavel, we need to inform the > client > >> > when > >> > > > the > >> > > > > > >>> task is > >> > > > > > >>> > > >> > > completed, > >> > > > > > >>> > > >> > > > > we > >> > > > > > >>> > > >> > > > > > > need > >> > > > > > >>> > > >> > > > > > > > > the > >> > > > > > >>> > > >> > > > > > > > > > ability to cancel the task. I see > >> > > several > >> > > > > ways > >> > > > > > >>> to > >> > > > > > >>> > > >> implement > >> > > > > > >>> > > >> > > > this: > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > 1. Сlient sends a request to the > >> > server > >> > > to > >> > > > > > >>> start a > >> > > > > > >>> > > task, > >> > > > > > >>> > > >> > > server > >> > > > > > >>> > > >> > > > > > > return > >> > > > > > >>> > > >> > > > > > > > > task > >> > > > > > >>> > > >> > > > > > > > > > id in response. Server notifies > >> client > >> > > > when > >> > > > > > >>> task is > >> > > > > > >>> > > >> > completed > >> > > > > > >>> > > >> > > > > with > >> > > > > > >>> > > >> > > > > > a > >> > > > > > >>> > > >> > > > > > > > new > >> > > > > > >>> > > >> > > > > > > > > > request (from server to client). > >> > Client > >> > > > can > >> > > > > > >>> cancel > >> > > > > > >>> > the > >> > > > > > >>> > > >> task > >> > > > > > >>> > > >> > > by > >> > > > > > >>> > > >> > > > > > > sending > >> > > > > > >>> > > >> > > > > > > > a > >> > > > > > >>> > > >> > > > > > > > > > new request with operation type > >> > "cancel" > >> > > > and > >> > > > > > >>> task > >> > > > > > >>> > id. > >> > > > > > >>> > > In > >> > > > > > >>> > > >> > this > >> > > > > > >>> > > >> > > > > case, > >> > > > > > >>> > > >> > > > > > > we > >> > > > > > >>> > > >> > > > > > > > > > should implement 2-ways requests. > >> > > > > > >>> > > >> > > > > > > > > > 2. Client generates unique task > id > >> and > >> > > > > sends a > >> > > > > > >>> > request > >> > > > > > >>> > > >> to > >> > > > > > >>> > > >> > the > >> > > > > > >>> > > >> > > > > > server > >> > > > > > >>> > > >> > > > > > > to > >> > > > > > >>> > > >> > > > > > > > > > start a task, server don't reply > >> > > > immediately > >> > > > > > but > >> > > > > > >>> > wait > >> > > > > > >>> > > >> until > >> > > > > > >>> > > >> > > > task > >> > > > > > >>> > > >> > > > > is > >> > > > > > >>> > > >> > > > > > > > > > completed. Client can cancel task > >> by > >> > > > sending > >> > > > > > new > >> > > > > > >>> > > request > >> > > > > > >>> > > >> > with > >> > > > > > >>> > > >> > > > > > > operation > >> > > > > > >>> > > >> > > > > > > > > > type "cancel" and task id. In > this > >> > case, > >> > > > we > >> > > > > > >>> should > >> > > > > > >>> > > >> decouple > >> > > > > > >>> > > >> > > > > request > >> > > > > > >>> > > >> > > > > > > and > >> > > > > > >>> > > >> > > > > > > > > > response on the server-side > >> (currently > >> > > > > > response > >> > > > > > >>> is > >> > > > > > >>> > > sent > >> > > > > > >>> > > >> > right > >> > > > > > >>> > > >> > > > > after > >> > > > > > >>> > > >> > > > > > > > > request > >> > > > > > >>> > > >> > > > > > > > > > was processed). Also, we can't be > >> sure > >> > > > that > >> > > > > > >>> task is > >> > > > > > >>> > > >> > > > successfully > >> > > > > > >>> > > >> > > > > > > > started > >> > > > > > >>> > > >> > > > > > > > > on > >> > > > > > >>> > > >> > > > > > > > > > a server. > >> > > > > > >>> > > >> > > > > > > > > > 3. Client sends a request to the > >> > server > >> > > to > >> > > > > > >>> start a > >> > > > > > >>> > > task, > >> > > > > > >>> > > >> > > server > >> > > > > > >>> > > >> > > > > > > return > >> > > > > > >>> > > >> > > > > > > > id > >> > > > > > >>> > > >> > > > > > > > > > in response. Client periodically > >> asks > >> > > the > >> > > > > > server > >> > > > > > >>> > about > >> > > > > > >>> > > >> task > >> > > > > > >>> > > >> > > > > status. > >> > > > > > >>> > > >> > > > > > > > > Client > >> > > > > > >>> > > >> > > > > > > > > > can cancel the task by sending > new > >> > > request > >> > > > > > with > >> > > > > > >>> > > >> operation > >> > > > > > >>> > > >> > > type > >> > > > > > >>> > > >> > > > > > > "cancel" > >> > > > > > >>> > > >> > > > > > > > > and > >> > > > > > >>> > > >> > > > > > > > > > task id. This case brings some > >> > overhead > >> > > to > >> > > > > the > >> > > > > > >>> > > >> > communication > >> > > > > > >>> > > >> > > > > > channel. > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Personally, I think that the case > >> with > >> > > > > 2-ways > >> > > > > > >>> > requests > >> > > > > > >>> > > >> is > >> > > > > > >>> > > >> > > > better, > >> > > > > > >>> > > >> > > > > > but > >> > > > > > >>> > > >> > > > > > > > I'm > >> > > > > > >>> > > >> > > > > > > > > > open to any other ideas. > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Aleksandr, > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Filtering logic for > >> > > > > > >>> OP_CLUSTER_GROUP_GET_NODE_IDS > >> > > > > > >>> > > looks > >> > > > > > >>> > > >> > > > > > > > overcomplicated. > >> > > > > > >>> > > >> > > > > > > > > Do > >> > > > > > >>> > > >> > > > > > > > > > we need server-side filtering at > >> all? > >> > > > > Wouldn't > >> > > > > > >>> it be > >> > > > > > >>> > > >> better > >> > > > > > >>> > > >> > > to > >> > > > > > >>> > > >> > > > > send > >> > > > > > >>> > > >> > > > > > > > basic > >> > > > > > >>> > > >> > > > > > > > > > info (ids, order, flags) for all > >> nodes > >> > > > > (there > >> > > > > > is > >> > > > > > >>> > > >> relatively > >> > > > > > >>> > > >> > > > small > >> > > > > > >>> > > >> > > > > > > > amount > >> > > > > > >>> > > >> > > > > > > > > of > >> > > > > > >>> > > >> > > > > > > > > > data) and extended info > >> (attributes) > >> > for > >> > > > > > >>> selected > >> > > > > > >>> > list > >> > > > > > >>> > > >> of > >> > > > > > >>> > > >> > > > nodes? > >> > > > > > >>> > > >> > > > > In > >> > > > > > >>> > > >> > > > > > > > this > >> > > > > > >>> > > >> > > > > > > > > > case, we can do basic node > >> filtration > >> > on > >> > > > > > >>> client-side > >> > > > > > >>> > > >> > > > > (forClients(), > >> > > > > > >>> > > >> > > > > > > > > > forServers(), forNodeIds(), > >> > forOthers(), > >> > > > > etc). > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Do you use standard ClusterNode > >> > > > > serialization? > >> > > > > > >>> There > >> > > > > > >>> > > are > >> > > > > > >>> > > >> > also > >> > > > > > >>> > > >> > > > > > metrics > >> > > > > > >>> > > >> > > > > > > > > > serialized with ClusterNode, do > we > >> > need > >> > > it > >> > > > > on > >> > > > > > >>> thin > >> > > > > > >>> > > >> client? > >> > > > > > >>> > > >> > > > There > >> > > > > > >>> > > >> > > > > > are > >> > > > > > >>> > > >> > > > > > > > > other > >> > > > > > >>> > > >> > > > > > > > > > interfaces exist to show > metrics, I > >> > > think > >> > > > > it's > >> > > > > > >>> > > >> redundant to > >> > > > > > >>> > > >> > > > > export > >> > > > > > >>> > > >> > > > > > > > > metrics > >> > > > > > >>> > > >> > > > > > > > > > to thin clients too. > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > What do you think? > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > пт, 22 нояб. 2019 г. в 20:15, > >> > Aleksandr > >> > > > > > Shapkin > >> > > > > > >>> < > >> > > > > > >>> > > >> > > > > lexw...@gmail.com > >> > > > > > >>> > > >> > > > > > >: > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Alex, > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > I think you can create a new > IEP > >> > page > >> > > > and > >> > > > > I > >> > > > > > >>> will > >> > > > > > >>> > > fill > >> > > > > > >>> > > >> it > >> > > > > > >>> > > >> > > with > >> > > > > > >>> > > >> > > > > the > >> > > > > > >>> > > >> > > > > > > > > Cluster > >> > > > > > >>> > > >> > > > > > > > > > > API details. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > In short, I’ve introduced > several > >> > new > >> > > > > codes: > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Cluster API is pretty > >> > straightforward: > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_IS_ACTIVE = 5000 > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_CHANGE_STATE = 5001 > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_CHANGE_WAL_STATE = > >> 5002 > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_GET_WAL_STATE = 5003 > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Cluster group codes: > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_GROUP_GET_NODE_IDS = > >> 5100 > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_GROUP_GET_NODE_INFO > = > >> > 5101 > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > The underlying implementation > is > >> > based > >> > > > on > >> > > > > > the > >> > > > > > >>> > thick > >> > > > > > >>> > > >> > client > >> > > > > > >>> > > >> > > > > logic. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > For every request, we provide a > >> > known > >> > > > > > topology > >> > > > > > >>> > > version > >> > > > > > >>> > > >> > and > >> > > > > > >>> > > >> > > if > >> > > > > > >>> > > >> > > > > it > >> > > > > > >>> > > >> > > > > > > has > >> > > > > > >>> > > >> > > > > > > > > > > changed, > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > a client updates it firstly and > >> then > >> > > > > > re-sends > >> > > > > > >>> the > >> > > > > > >>> > > >> > filtering > >> > > > > > >>> > > >> > > > > > > request. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Alongside the topVer a client > >> sends > >> > a > >> > > > > > >>> serialized > >> > > > > > >>> > > nodes > >> > > > > > >>> > > >> > > > > projection > >> > > > > > >>> > > >> > > > > > > > > object > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > that could be considered as a > >> code > >> > to > >> > > > > value > >> > > > > > >>> > mapping. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Consider: [{Code = 1, Value= > >> > > [“DotNet”, > >> > > > > > >>> > > >> “MyAttribute”}, > >> > > > > > >>> > > >> > > > > {Code=2, > >> > > > > > >>> > > >> > > > > > > > > > Value=1}] > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Where “1” stands for Attribute > >> > > filtering > >> > > > > and > >> > > > > > >>> “2” – > >> > > > > > >>> > > >> > > > > > serverNodesOnly > >> > > > > > >>> > > >> > > > > > > > > flag. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > As a result of request > >> processing, a > >> > > > > server > >> > > > > > >>> sends > >> > > > > > >>> > > >> nodeId > >> > > > > > >>> > > >> > > > UUIDs > >> > > > > > >>> > > >> > > > > > and > >> > > > > > >>> > > >> > > > > > > a > >> > > > > > >>> > > >> > > > > > > > > > > current topVer. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > When a client obtains nodeIds, > it > >> > can > >> > > > > > perform > >> > > > > > >>> a > >> > > > > > >>> > > >> NODE_INFO > >> > > > > > >>> > > >> > > > call > >> > > > > > >>> > > >> > > > > to > >> > > > > > >>> > > >> > > > > > > > get a > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > serialized ClusterNode object. > In > >> > > > addition > >> > > > > > >>> there > >> > > > > > >>> > > >> should > >> > > > > > >>> > > >> > be > >> > > > > > >>> > > >> > > a > >> > > > > > >>> > > >> > > > > > > > different > >> > > > > > >>> > > >> > > > > > > > > > API > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > method for accessing/updating > >> node > >> > > > > metrics. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > чт, 21 нояб. 2019 г. в 12:32, > >> Sergey > >> > > > > Kozlov > >> > > > > > < > >> > > > > > >>> > > >> > > > > > skoz...@gridgain.com > >> > > > > > >>> > > >> > > > > > > >: > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > Hi Pavel > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > On Thu, Nov 21, 2019 at 11:30 > >> AM > >> > > Pavel > >> > > > > > >>> Tupitsyn > >> > > > > > >>> > < > >> > > > > > >>> > > >> > > > > > > > > ptupit...@apache.org> > >> > > > > > >>> > > >> > > > > > > > > > > > wrote: > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > 1. I believe that Cluster > >> > > operations > >> > > > > for > >> > > > > > >>> Thin > >> > > > > > >>> > > >> Client > >> > > > > > >>> > > >> > > > > protocol > >> > > > > > >>> > > >> > > > > > > are > >> > > > > > >>> > > >> > > > > > > > > > > already > >> > > > > > >>> > > >> > > > > > > > > > > > > in the works > >> > > > > > >>> > > >> > > > > > > > > > > > > by Alexandr Shapkin. Can't > >> find > >> > > the > >> > > > > > ticket > >> > > > > > >>> > > though. > >> > > > > > >>> > > >> > > > > > > > > > > > > Alexandr, can you please > >> confirm > >> > > and > >> > > > > > >>> attach > >> > > > > > >>> > the > >> > > > > > >>> > > >> > ticket > >> > > > > > >>> > > >> > > > > > number? > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > 2. Proposed changes will > work > >> > only > >> > > > for > >> > > > > > >>> Java > >> > > > > > >>> > > tasks > >> > > > > > >>> > > >> > that > >> > > > > > >>> > > >> > > > are > >> > > > > > >>> > > >> > > > > > > > already > >> > > > > > >>> > > >> > > > > > > > > > > > deployed > >> > > > > > >>> > > >> > > > > > > > > > > > > on server nodes. > >> > > > > > >>> > > >> > > > > > > > > > > > > This is mostly useless for > >> other > >> > > > thin > >> > > > > > >>> clients > >> > > > > > >>> > we > >> > > > > > >>> > > >> have > >> > > > > > >>> > > >> > > > > > (Python, > >> > > > > > >>> > > >> > > > > > > > PHP, > >> > > > > > >>> > > >> > > > > > > > > > > .NET, > >> > > > > > >>> > > >> > > > > > > > > > > > > C++). > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > I don't guess so. The task > >> > > (execution) > >> > > > > is > >> > > > > > a > >> > > > > > >>> way > >> > > > > > >>> > to > >> > > > > > >>> > > >> > > > implement > >> > > > > > >>> > > >> > > > > > own > >> > > > > > >>> > > >> > > > > > > > > layer > >> > > > > > >>> > > >> > > > > > > > > > > for > >> > > > > > >>> > > >> > > > > > > > > > > > the thin client application. > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > We should think of a way to > >> make > >> > > > this > >> > > > > > >>> useful > >> > > > > > >>> > for > >> > > > > > >>> > > >> all > >> > > > > > >>> > > >> > > > > clients. > >> > > > > > >>> > > >> > > > > > > > > > > > > For example, we may allow > >> > sending > >> > > > > tasks > >> > > > > > in > >> > > > > > >>> > some > >> > > > > > >>> > > >> > > scripting > >> > > > > > >>> > > >> > > > > > > > language > >> > > > > > >>> > > >> > > > > > > > > > like > >> > > > > > >>> > > >> > > > > > > > > > > > > Javascript. > >> > > > > > >>> > > >> > > > > > > > > > > > > Thoughts? > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > The arbitrary code execution > >> from > >> > a > >> > > > > remote > >> > > > > > >>> > client > >> > > > > > >>> > > >> must > >> > > > > > >>> > > >> > be > >> > > > > > >>> > > >> > > > > > > protected > >> > > > > > >>> > > >> > > > > > > > > > > > from malicious code. > >> > > > > > >>> > > >> > > > > > > > > > > > I don't know how it could be > >> > > designed > >> > > > > but > >> > > > > > >>> > without > >> > > > > > >>> > > >> that > >> > > > > > >>> > > >> > we > >> > > > > > >>> > > >> > > > > open > >> > > > > > >>> > > >> > > > > > > the > >> > > > > > >>> > > >> > > > > > > > > hole > >> > > > > > >>> > > >> > > > > > > > > > > to > >> > > > > > >>> > > >> > > > > > > > > > > > kill cluster. > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > On Thu, Nov 21, 2019 at > >> 11:21 AM > >> > > > > Sergey > >> > > > > > >>> > Kozlov < > >> > > > > > >>> > > >> > > > > > > > > skoz...@gridgain.com > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > wrote: > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > Hi Alex > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > The idea is great. But I > >> have > >> > > some > >> > > > > > >>> concerns > >> > > > > > >>> > > that > >> > > > > > >>> > > >> > > > probably > >> > > > > > >>> > > >> > > > > > > > should > >> > > > > > >>> > > >> > > > > > > > > be > >> > > > > > >>> > > >> > > > > > > > > > > > taken > >> > > > > > >>> > > >> > > > > > > > > > > > > > into account for design: > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > 1. We need to have the > >> > > ability > >> > > > to > >> > > > > > >>> stop a > >> > > > > > >>> > > task > >> > > > > > >>> > > >> > > > > execution, > >> > > > > > >>> > > >> > > > > > > > smth > >> > > > > > >>> > > >> > > > > > > > > > like > >> > > > > > >>> > > >> > > > > > > > > > > > > > OP_COMPUTE_CANCEL_TASK > >> > > > operation > >> > > > > > >>> (client > >> > > > > > >>> > > to > >> > > > > > >>> > > >> > > server) > >> > > > > > >>> > > >> > > > > > > > > > > > > > 2. What's about task > >> > > execution > >> > > > > > >>> timeout? > >> > > > > > >>> > It > >> > > > > > >>> > > >> may > >> > > > > > >>> > > >> > > help > >> > > > > > >>> > > >> > > > to > >> > > > > > >>> > > >> > > > > > the > >> > > > > > >>> > > >> > > > > > > > > > cluster > >> > > > > > >>> > > >> > > > > > > > > > > > > > survival for buggy > tasks > >> > > > > > >>> > > >> > > > > > > > > > > > > > 3. Ignite doesn't have > >> > > > > > >>> > roles/authorization > >> > > > > > >>> > > >> > > > > functionality > >> > > > > > >>> > > >> > > > > > > for > >> > > > > > >>> > > >> > > > > > > > > > now. > >> > > > > > >>> > > >> > > > > > > > > > > > But > >> > > > > > >>> > > >> > > > > > > > > > > > > a > >> > > > > > >>> > > >> > > > > > > > > > > > > > task is the risky > >> operation > >> > > for > >> > > > > > >>> cluster > >> > > > > > >>> > > (for > >> > > > > > >>> > > >> > > > security > >> > > > > > >>> > > >> > > > > > > > > reasons). > >> > > > > > >>> > > >> > > > > > > > > > > > Could > >> > > > > > >>> > > >> > > > > > > > > > > > > we > >> > > > > > >>> > > >> > > > > > > > > > > > > > add for Ignite > >> > configuration > >> > > > new > >> > > > > > >>> options: > >> > > > > > >>> > > >> > > > > > > > > > > > > > - Explicit turning > on > >> > for > >> > > > > > compute > >> > > > > > >>> task > >> > > > > > >>> > > >> > support > >> > > > > > >>> > > >> > > > for > >> > > > > > >>> > > >> > > > > > thin > >> > > > > > >>> > > >> > > > > > > > > > > protocol > >> > > > > > >>> > > >> > > > > > > > > > > > > > (disabled by > default) > >> > for > >> > > > > whole > >> > > > > > >>> > cluster > >> > > > > > >>> > > >> > > > > > > > > > > > > > - Explicit turning > on > >> > for > >> > > > > > compute > >> > > > > > >>> task > >> > > > > > >>> > > >> > support > >> > > > > > >>> > > >> > > > for > >> > > > > > >>> > > >> > > > > a > >> > > > > > >>> > > >> > > > > > > node > >> > > > > > >>> > > >> > > > > > > > > > > > > > - The list of task > >> names > >> > > > > > (classes) > >> > > > > > >>> > > >> allowed to > >> > > > > > >>> > > >> > > > > execute > >> > > > > > >>> > > >> > > > > > > by > >> > > > > > >>> > > >> > > > > > > > > thin > >> > > > > > >>> > > >> > > > > > > > > > > > > client. > >> > > > > > >>> > > >> > > > > > > > > > > > > > 4. Support the > labeling > >> for > >> > > > task > >> > > > > > >>> that may > >> > > > > > >>> > > >> help > >> > > > > > >>> > > >> > to > >> > > > > > >>> > > >> > > > > > > > investigate > >> > > > > > >>> > > >> > > > > > > > > > > issues > >> > > > > > >>> > > >> > > > > > > > > > > > > on > >> > > > > > >>> > > >> > > > > > > > > > > > > > cluster (the idea from > >> > IEP-34 > >> > > > > [1]) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > 1. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > >> > > > > > >>> > > > >> > > > > > >>> > > >> > > > > > >>> > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > On Thu, Nov 21, 2019 at > >> 10:58 > >> > AM > >> > > > > Alex > >> > > > > > >>> > > Plehanov < > >> > > > > > >>> > > >> > > > > > > > > > > > plehanov.a...@gmail.com> > >> > > > > > >>> > > >> > > > > > > > > > > > > > wrote: > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Hello, Igniters! > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > I have plans to start > >> > > > > implementation > >> > > > > > >>> of > >> > > > > > >>> > > >> Compute > >> > > > > > >>> > > >> > > > > interface > >> > > > > > >>> > > >> > > > > > > for > >> > > > > > >>> > > >> > > > > > > > > > > Ignite > >> > > > > > >>> > > >> > > > > > > > > > > > > thin > >> > > > > > >>> > > >> > > > > > > > > > > > > > > client and want to > >> discuss > >> > > > > features > >> > > > > > >>> that > >> > > > > > >>> > > >> should > >> > > > > > >>> > > >> > be > >> > > > > > >>> > > >> > > > > > > > implemented. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > We already have Compute > >> > > > > > >>> implementation for > >> > > > > > >>> > > >> > > > binary-rest > >> > > > > > >>> > > >> > > > > > > > clients > >> > > > > > >>> > > >> > > > > > > > > > > > > > > (GridClientCompute), > >> which > >> > > have > >> > > > > the > >> > > > > > >>> > > following > >> > > > > > >>> > > >> > > > > > > functionality: > >> > > > > > >>> > > >> > > > > > > > > > > > > > > - Filtering cluster > nodes > >> > > > > > >>> (projection) for > >> > > > > > >>> > > >> > compute > >> > > > > > >>> > > >> > > > > > > > > > > > > > > - Executing task by the > >> name > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > I think we can > implement > >> > this > >> > > > > > >>> > functionality > >> > > > > > >>> > > >> in a > >> > > > > > >>> > > >> > > thin > >> > > > > > >>> > > >> > > > > > > client > >> > > > > > >>> > > >> > > > > > > > as > >> > > > > > >>> > > >> > > > > > > > > > > well. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > First of all, we need > >> some > >> > > > > operation > >> > > > > > >>> types > >> > > > > > >>> > > to > >> > > > > > >>> > > >> > > > request a > >> > > > > > >>> > > >> > > > > > > list > >> > > > > > >>> > > >> > > > > > > > of > >> > > > > > >>> > > >> > > > > > > > > > all > >> > > > > > >>> > > >> > > > > > > > > > > > > > > available nodes and > >> probably > >> > > > node > >> > > > > > >>> > attributes > >> > > > > > >>> > > >> (by > >> > > > > > >>> > > >> > a > >> > > > > > >>> > > >> > > > list > >> > > > > > >>> > > >> > > > > > of > >> > > > > > >>> > > >> > > > > > > > > > nodes). > >> > > > > > >>> > > >> > > > > > > > > > > > Node > >> > > > > > >>> > > >> > > > > > > > > > > > > > > attributes will be > >> helpful > >> > if > >> > > we > >> > > > > > will > >> > > > > > >>> > decide > >> > > > > > >>> > > >> to > >> > > > > > >>> > > >> > > > > implement > >> > > > > > >>> > > >> > > > > > > > > analog > >> > > > > > >>> > > >> > > > > > > > > > of > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> ClusterGroup#forAttribute or > >> > > > > > >>> > > >> > > > ClusterGroup#forePredicate > >> > > > > > >>> > > >> > > > > > > > methods > >> > > > > > >>> > > >> > > > > > > > > > in > >> > > > > > >>> > > >> > > > > > > > > > > > the > >> > > > > > >>> > > >> > > > > > > > > > > > > > thin > >> > > > > > >>> > > >> > > > > > > > > > > > > > > client. Perhaps they > can > >> be > >> > > > > > requested > >> > > > > > >>> > > lazily. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > From the protocol point > >> of > >> > > view > >> > > > > > there > >> > > > > > >>> will > >> > > > > > >>> > > be > >> > > > > > >>> > > >> two > >> > > > > > >>> > > >> > > new > >> > > > > > >>> > > >> > > > > > > > > operations: > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > OP_CLUSTER_GET_NODES > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: empty > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: long > >> > > topologyVersion, > >> > > > > int > >> > > > > > >>> > > >> > > > > minorTopologyVersion, > >> > > > > > >>> > > >> > > > > > > int > >> > > > > > >>> > > >> > > > > > > > > > > > > nodesCount, > >> > > > > > >>> > > >> > > > > > > > > > > > > > > for each node set of > node > >> > > fields > >> > > > > > (UUID > >> > > > > > >>> > > nodeId, > >> > > > > > >>> > > >> > > Object > >> > > > > > >>> > > >> > > > > or > >> > > > > > >>> > > >> > > > > > > > String > >> > > > > > >>> > > >> > > > > > > > > > > > > > > consistentId, long > order, > >> > etc) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > OP_CLUSTER_GET_NODE_ATTRIBUTES > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: int > nodesCount, > >> for > >> > > > each > >> > > > > > >>> node: > >> > > > > > >>> > UUID > >> > > > > > >>> > > >> > nodeId > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: int > nodesCount, > >> > for > >> > > > each > >> > > > > > >>> node: > >> > > > > > >>> > int > >> > > > > > >>> > > >> > > > > > > attributesCount, > >> > > > > > >>> > > >> > > > > > > > > for > >> > > > > > >>> > > >> > > > > > > > > > > > each > >> > > > > > >>> > > >> > > > > > > > > > > > > > node > >> > > > > > >>> > > >> > > > > > > > > > > > > > > attribute: String name, > >> > Object > >> > > > > value > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > To execute tasks we > need > >> > > > something > >> > > > > > >>> like > >> > > > > > >>> > > these > >> > > > > > >>> > > >> > > methods > >> > > > > > >>> > > >> > > > > in > >> > > > > > >>> > > >> > > > > > > the > >> > > > > > >>> > > >> > > > > > > > > > client > >> > > > > > >>> > > >> > > > > > > > > > > > > API: > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Object execute(String > >> task, > >> > > > Object > >> > > > > > >>> arg) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Future<Object> > >> > > > executeAsync(String > >> > > > > > >>> task, > >> > > > > > >>> > > >> Object > >> > > > > > >>> > > >> > > arg) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Object > >> > affinityExecute(String > >> > > > > task, > >> > > > > > >>> String > >> > > > > > >>> > > >> cache, > >> > > > > > >>> > > >> > > > > Object > >> > > > > > >>> > > >> > > > > > > key, > >> > > > > > >>> > > >> > > > > > > > > > > Object > >> > > > > > >>> > > >> > > > > > > > > > > > > arg) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Future<Object> > >> > > > > > >>> affinityExecuteAsync(String > >> > > > > > >>> > > >> task, > >> > > > > > >>> > > >> > > > String > >> > > > > > >>> > > >> > > > > > > > cache, > >> > > > > > >>> > > >> > > > > > > > > > > Object > >> > > > > > >>> > > >> > > > > > > > > > > > > > key, > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Object arg) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Which can be mapped to > >> > > protocol > >> > > > > > >>> > operations: > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > OP_COMPUTE_EXECUTE_TASK > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: UUID nodeId, > >> String > >> > > > > > taskName, > >> > > > > > >>> > > Object > >> > > > > > >>> > > >> arg > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: Object result > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: String > >> cacheName, > >> > > > Object > >> > > > > > key, > >> > > > > > >>> > > String > >> > > > > > >>> > > >> > > > taskName, > >> > > > > > >>> > > >> > > > > > > > Object > >> > > > > > >>> > > >> > > > > > > > > > arg > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: Object result > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > The second operation is > >> > needed > >> > > > > > >>> because we > >> > > > > > >>> > > >> > sometimes > >> > > > > > >>> > > >> > > > > can't > >> > > > > > >>> > > >> > > > > > > > > > calculate > >> > > > > > >>> > > >> > > > > > > > > > > > and > >> > > > > > >>> > > >> > > > > > > > > > > > > > > connect to affinity > node > >> on > >> > > the > >> > > > > > >>> > client-side > >> > > > > > >>> > > >> > > (affinity > >> > > > > > >>> > > >> > > > > > > > awareness > >> > > > > > >>> > > >> > > > > > > > > > can > >> > > > > > >>> > > >> > > > > > > > > > > > be > >> > > > > > >>> > > >> > > > > > > > > > > > > > > disabled, custom > affinity > >> > > > function > >> > > > > > >>> can be > >> > > > > > >>> > > >> used or > >> > > > > > >>> > > >> > > > there > >> > > > > > >>> > > >> > > > > > can > >> > > > > > >>> > > >> > > > > > > > be > >> > > > > > >>> > > >> > > > > > > > > no > >> > > > > > >>> > > >> > > > > > > > > > > > > > > connection between > client > >> > and > >> > > > > > affinity > >> > > > > > >>> > > node), > >> > > > > > >>> > > >> but > >> > > > > > >>> > > >> > > we > >> > > > > > >>> > > >> > > > > can > >> > > > > > >>> > > >> > > > > > > make > >> > > > > > >>> > > >> > > > > > > > > > best > >> > > > > > >>> > > >> > > > > > > > > > > > > effort > >> > > > > > >>> > > >> > > > > > > > > > > > > > > to send request to > target > >> > node > >> > > > if > >> > > > > > >>> affinity > >> > > > > > >>> > > >> > > awareness > >> > > > > > >>> > > >> > > > is > >> > > > > > >>> > > >> > > > > > > > > enabled. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Currently, on the > >> > server-side > >> > > > > > requests > >> > > > > > >>> > > always > >> > > > > > >>> > > >> > > > processed > >> > > > > > >>> > > >> > > > > > > > > > > synchronously > >> > > > > > >>> > > >> > > > > > > > > > > > > and > >> > > > > > >>> > > >> > > > > > > > > > > > > > > responses are sent > right > >> > after > >> > > > > > >>> request was > >> > > > > > >>> > > >> > > processed. > >> > > > > > >>> > > >> > > > > To > >> > > > > > >>> > > >> > > > > > > > > execute > >> > > > > > >>> > > >> > > > > > > > > > > long > >> > > > > > >>> > > >> > > > > > > > > > > > > > tasks > >> > > > > > >>> > > >> > > > > > > > > > > > > > > async we should whether > >> > change > >> > > > > this > >> > > > > > >>> logic > >> > > > > > >>> > or > >> > > > > > >>> > > >> > > > introduce > >> > > > > > >>> > > >> > > > > > some > >> > > > > > >>> > > >> > > > > > > > > kind > >> > > > > > >>> > > >> > > > > > > > > > > > > two-way > >> > > > > > >>> > > >> > > > > > > > > > > > > > > communication between > >> client > >> > > and > >> > > > > > >>> server > >> > > > > > >>> > (now > >> > > > > > >>> > > >> only > >> > > > > > >>> > > >> > > > > one-way > >> > > > > > >>> > > >> > > > > > > > > > requests > >> > > > > > >>> > > >> > > > > > > > > > > > from > >> > > > > > >>> > > >> > > > > > > > > > > > > > > client to server are > >> > allowed). > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Two-way communication > can > >> > also > >> > > > be > >> > > > > > >>> useful > >> > > > > > >>> > in > >> > > > > > >>> > > >> the > >> > > > > > >>> > > >> > > > future > >> > > > > > >>> > > >> > > > > if > >> > > > > > >>> > > >> > > > > > > we > >> > > > > > >>> > > >> > > > > > > > > will > >> > > > > > >>> > > >> > > > > > > > > > > > send > >> > > > > > >>> > > >> > > > > > > > > > > > > > some > >> > > > > > >>> > > >> > > > > > > > > > > > > > > server-side generated > >> events > >> > > to > >> > > > > > >>> clients. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > In case of two-way > >> > > communication > >> > > > > > >>> there can > >> > > > > > >>> > > be > >> > > > > > >>> > > >> new > >> > > > > > >>> > > >> > > > > > > operations > >> > > > > > >>> > > >> > > > > > > > > > > > > introduced: > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > OP_COMPUTE_EXECUTE_TASK > >> > (from > >> > > > > client > >> > > > > > >>> to > >> > > > > > >>> > > >> server) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: UUID nodeId, > >> String > >> > > > > > taskName, > >> > > > > > >>> > > Object > >> > > > > > >>> > > >> arg > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: long taskId > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > OP_COMPUTE_TASK_FINISHED > >> > (from > >> > > > > > server > >> > > > > > >>> to > >> > > > > > >>> > > >> client) > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: taskId, Object > >> > result > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: empty > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > The same for affinity > >> > > requests. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Also, we can implement > >> not > >> > > only > >> > > > > > >>> execute > >> > > > > > >>> > task > >> > > > > > >>> > > >> > > > operation, > >> > > > > > >>> > > >> > > > > > but > >> > > > > > >>> > > >> > > > > > > > > some > >> > > > > > >>> > > >> > > > > > > > > > > > other > >> > > > > > >>> > > >> > > > > > > > > > > > > > > operations from > >> > IgniteCompute > >> > > > > > >>> (broadcast, > >> > > > > > >>> > > run, > >> > > > > > >>> > > >> > > call), > >> > > > > > >>> > > >> > > > > but > >> > > > > > >>> > > >> > > > > > > it > >> > > > > > >>> > > >> > > > > > > > > will > >> > > > > > >>> > > >> > > > > > > > > > > be > >> > > > > > >>> > > >> > > > > > > > > > > > > > useful > >> > > > > > >>> > > >> > > > > > > > > > > > > > > only for java thin > >> client. > >> > And > >> > > > > even > >> > > > > > >>> with > >> > > > > > >>> > > java > >> > > > > > >>> > > >> > thin > >> > > > > > >>> > > >> > > > > client > >> > > > > > >>> > > >> > > > > > > we > >> > > > > > >>> > > >> > > > > > > > > > should > >> > > > > > >>> > > >> > > > > > > > > > > > > > whether > >> > > > > > >>> > > >> > > > > > > > > > > > > > > implement > >> peer-class-loading > >> > > for > >> > > > > > thin > >> > > > > > >>> > > clients > >> > > > > > >>> > > >> > (this > >> > > > > > >>> > > >> > > > > also > >> > > > > > >>> > > >> > > > > > > > > requires > >> > > > > > >>> > > >> > > > > > > > > > > > > two-way > >> > > > > > >>> > > >> > > > > > > > > > > > > > > client-server > >> communication) > >> > > or > >> > > > > put > >> > > > > > >>> > classes > >> > > > > > >>> > > >> with > >> > > > > > >>> > > >> > > > > executed > >> > > > > > >>> > > >> > > > > > > > > > closures > >> > > > > > >>> > > >> > > > > > > > > > > to > >> > > > > > >>> > > >> > > > > > > > > > > > > the > >> > > > > > >>> > > >> > > > > > > > > > > > > > > server locally. > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > What do you think about > >> > > proposed > >> > > > > > >>> protocol > >> > > > > > >>> > > >> > changes? > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Do we need two-way > >> requests > >> > > > > between > >> > > > > > >>> client > >> > > > > > >>> > > and > >> > > > > > >>> > > >> > > > server? > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Do we need support of > >> > compute > >> > > > > > methods > >> > > > > > >>> > other > >> > > > > > >>> > > >> than > >> > > > > > >>> > > >> > > > > "execute > >> > > > > > >>> > > >> > > > > > > > > task"? > >> > > > > > >>> > > >> > > > > > > > > > > > > > > What do you think about > >> > > > > > >>> peer-class-loading > >> > > > > > >>> > > for > >> > > > > > >>> > > >> > thin > >> > > > > > >>> > > >> > > > > > > clients? > >> > > > > > >>> > > >> > > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > -- > >> > > > > > >>> > > >> > > > > > > > > > > > > > Sergey Kozlov > >> > > > > > >>> > > >> > > > > > > > > > > > > > GridGain Systems > >> > > > > > >>> > > >> > > > > > > > > > > > > > www.gridgain.com > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > -- > >> > > > > > >>> > > >> > > > > > > > > > > > Sergey Kozlov > >> > > > > > >>> > > >> > > > > > > > > > > > GridGain Systems > >> > > > > > >>> > > >> > > > > > > > > > > > www.gridgain.com > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > -- > >> > > > > > >>> > > >> > > > > > > > > > > Alex. > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > >> > > > > > >>> > > > > >> > > > > > >>> > > > >> > > > > > >>> > > >> > > > > > >>> > >> > > > > > >> > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
