Re: [infinispan-dev] Design of Remote Hot Rod events - round 2
On Dec 13, 2013, at 4:33 PM, Radim Vansa rva...@redhat.com wrote: On 12/13/2013 03:49 PM, Galder Zamarreño wrote: On Dec 6, 2013, at 4:17 PM, Radim Vansa rva...@redhat.com wrote: Btw., when Hot Rod fails to hit the primary owner, should the non-owner propagate the SourceID to primary owner somehow? Or is in this case acceptable notifying the listener about its own change? If the call lands in a non-owner, it's probably simpler for the non-owner to send the notification there and then. ACK information tracking would probably be distributed, in which case it'd need to deal with potential failure of the primary owner. I don't think I understand that properly. The node responsible for notifying the client is either primary owner, or operation origin (where the write has landed in fact). Previously, we were saying that the responsible node should be the primary owner - now you say that the origin. When the cluster is accessed only remotely, it does not have much performance impact (as these two are mostly the same node), but with cluster in compatibility mode the decision could affect the performance a lot. Normally, a Hot Rod call would land on the owner of the key, which then is able to send the notification itself. The question, as you've rightly asked, is what to do when the call lands in a non-owner. Is that because the owner node has failed? Is it because the client has a flaky issue with the hash function and not all calls are directed to non-owners? Rather than spending time forcing the owner to be the one that sends the notification, a non-owner might say: something weird happened here, I should have received this call, but just in case, I'll send notifications linked to it. So, do you think that this should be the origin (easier to implement, with access to distributed ack registry it can retrieve the information, but with higher latency as the ack info is probably affine to the entry itself) or primary owner (in this case you'd have to propagate the source ID with the write command). Btw., what should be the source ID for operations coming from library mode? JGroups address of the node? Personally, I think remote eventing is complex enough as it is to add compatibility in this first release because there's no event infrastructure for memcached nor REST. We should focus on getting remote events right for Hot Rod. Radim Cheers, -- Galder Zamarreño gal...@redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org -- Radim Vansa rva...@redhat.com JBoss DataGrid QA -- Galder Zamarreño gal...@redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Design of Remote Hot Rod events - round 2
On Dec 13, 2013, at 5:08 PM, Emmanuel Bernard emman...@hibernate.org wrote:
Much better. Some more feedback.
== Filter and converter
I am wondering if there is a benefit in separating filters and
converters. It does add some understanding complexity so a single
ServerListener with the methods from RemoteFilter / RemoveConverter
might be better.
I think the two are fairly tight in, so I think that might be a good idea.
Should filter / converter impls have a @FilterId() / @ConverterId or
should even the id be configured as late binding?
That falls into the configuration part which I hoped @Tristan would provide
some insight on how we'd plug these filter/convert impls into the server. If
they're defined via Infinispan configuration, the ID could be provided via
there. If they're found via service loader pattern, annotations could be used,
or callback methods. The former is probably more elegant.
== Custom event contents
I an not following, is the custom content always received as byte[] by
the client? Or is the generic parameter K in
RemoteCacheEntryCustomEventK the actual return type of getEventData() ?
I'd love (in Java) to declare the returned type of the converter via
generics in the RemoteConverter impl (class SomeRCString extends
RemoteConverterString {}) and somehow use that information on
the client side.
Event data is a byte[] as formatted by the converter on the server side. On the
client side, I can only expose this as is, as byte[]. The callback would then
need to disect it somehow.
The K in RemoteCacheEntryCustomEventK should be removed really, since with
these custom event calls, there's no key provided.
== Example of continuous query atop remote listeners
Thinking about how to implement continuous query atop this
infrastructure I am missing a few things.
The primary problem is that I don't want to enlist a filter id per
continuous query I want to run. Not only that but I'd love to be able to
add a continuous query on the fly and disable it on the fly as well per
client. For that filters and converters are not flexible enough.
What is missing is the ability to pass parameters from the client to
the remote filter and remote converter. Parameters should be provided
*per client*. Say Client 1 register the continuous query listener with
where age 19 and client 2 registers the CQ listener with where name
= emmanuel. The filter knowing for which client it filters, it will be able
to only
return the keys that match the query.
This all sounds a bit like remote code exectution to me? You're asking for the
client to pass some kind of executable thing that acts as a filter. That's a
separate feature IMO, which I believe @Tristan is looking into. Once that's in
place, I'm happy to enhance stuff in the remote event side to support it.
From a remote event support perspective, server side static filter/converters
is as far as I'd like to take it.
Another useful but not fundamental notion is the ability for a client to
enlist the same filter id / converter id tuple with different
parameters. The same client might be interested in several different
queries.
BTW have you considered some kind of multiplexing mechanism in case the
several client listeners on the same client are interested in the same
event?
If they're in the same client, they'd have the same source ID, so just sending
it once would be doable, assuming they're not using different converters.
Cheers,
Emmanuel
On Thu 2013-12-05 17:16, Galder Zamarreño wrote:
Hi all,
Re: https://github.com/infinispan/infinispan/wiki/Remote-Hot-Rod-Events
Thanks a lot for the feedback provided in last thread. It was very
constructive feedback :)
I've just finished updating the design document with the feedback provided
in the previous email thread. Can you please have another read and let the
list know what you think of it?
Side note: The scope has got bigger (with the addition of
filters/converters), so we might need to consider whether we want all
features in next version, or whether some parts could be branched out to
next iterations.
Cheers,
--
Galder Zamarreño
gal...@redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
gal...@redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Design of Remote Hot Rod events - round 2
On Thu 2013-12-19 9:46, Galder Zamarreño wrote: == Example of continuous query atop remote listeners Thinking about how to implement continuous query atop this infrastructure I am missing a few things. The primary problem is that I don't want to enlist a filter id per continuous query I want to run. Not only that but I'd love to be able to add a continuous query on the fly and disable it on the fly as well per client. For that filters and converters are not flexible enough. What is missing is the ability to pass parameters from the client to the remote filter and remote converter. Parameters should be provided *per client*. Say Client 1 register the continuous query listener with where age 19 and client 2 registers the CQ listener with where name = emmanuel. The filter knowing for which client it filters, it will be able to only return the keys that match the query. This all sounds a bit like remote code exectution to me? You're asking for the client to pass some kind of executable thing that acts as a filter. That's a separate feature IMO, which I believe @Tristan is looking into. Once that's in place, I'm happy to enhance stuff in the remote event side to support it. I don't think you are correct. This is not remote execution in the sense of arbitrary code driven by the client. Remote execution will likely be triggered, render a result and stop. It will not send matching events in a continuous fashion. Plus remote execution will likely involve dynamic languages and I'm not sure we want to go that route for things like continuous query. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Design of Remote Hot Rod events - round 2
Hi Galder, regarding the Client Identification paragraph I was thinking of the connection there might be with authenticated session ids I describe in the security document [1] in order to reduce the potential proliferation of identifiers. In the security case it is the server who is generating a unique session identifier at the end of a successful auth handshake. Such an identifier is then sent back from the client for all subsequent requests to avoid re-authentication. My plan was to tie this session id merely to the user's principal but this would not allow recognizing a dropped/restarted client. My proposal is therefore that the HotRod protocol should add just one element which can serve both purposes. Tristan [1] https://github.com/infinispan/infinispan/wiki/Security On 12/05/2013 05:16 PM, Galder Zamarreño wrote: Hi all, Re: https://github.com/infinispan/infinispan/wiki/Remote-Hot-Rod-Events Thanks a lot for the feedback provided in last thread. It was very constructive feedback :) I've just finished updating the design document with the feedback provided in the previous email thread. Can you please have another read and let the list know what you think of it? Side note: The scope has got bigger (with the addition of filters/converters), so we might need to consider whether we want all features in next version, or whether some parts could be branched out to next iterations. Cheers, -- Galder Zamarreño gal...@redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Design of Remote Hot Rod events - round 2
On Thu, Dec 19, 2013 at 2:15 PM, Emmanuel Bernard emman...@hibernate.orgwrote: On Thu 2013-12-19 9:46, Galder Zamarreño wrote: == Example of continuous query atop remote listeners Thinking about how to implement continuous query atop this infrastructure I am missing a few things. The primary problem is that I don't want to enlist a filter id per continuous query I want to run. Not only that but I'd love to be able to add a continuous query on the fly and disable it on the fly as well per client. For that filters and converters are not flexible enough. What is missing is the ability to pass parameters from the client to the remote filter and remote converter. Parameters should be provided *per client*. Say Client 1 register the continuous query listener with where age 19 and client 2 registers the CQ listener with where name = emmanuel. The filter knowing for which client it filters, it will be able to only return the keys that match the query. This all sounds a bit like remote code exectution to me? You're asking for the client to pass some kind of executable thing that acts as a filter. That's a separate feature IMO, which I believe @Tristan is looking into. Once that's in place, I'm happy to enhance stuff in the remote event side to support it. I don't think you are correct. This is not remote execution in the sense of arbitrary code driven by the client. Remote execution will likely be triggered, render a result and stop. It will not send matching events in a continuous fashion. Plus remote execution will likely involve dynamic languages and I'm not sure we want to go that route for things like continuous query. To be clear, this is exactly the same as the filter parameters that Radim was asking for, right? From Infinispan's point of view, the filter just takes a String parameter, and the fact that that string can be parsed by the filter in a particular language is irrelevant. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Design of Remote Hot Rod events - round 2
On Thu 2013-12-19 19:57, Dan Berindei wrote: On Thu, Dec 19, 2013 at 2:15 PM, Emmanuel Bernard emman...@hibernate.orgwrote: On Thu 2013-12-19 9:46, Galder Zamarreño wrote: == Example of continuous query atop remote listeners Thinking about how to implement continuous query atop this infrastructure I am missing a few things. The primary problem is that I don't want to enlist a filter id per continuous query I want to run. Not only that but I'd love to be able to add a continuous query on the fly and disable it on the fly as well per client. For that filters and converters are not flexible enough. What is missing is the ability to pass parameters from the client to the remote filter and remote converter. Parameters should be provided *per client*. Say Client 1 register the continuous query listener with where age 19 and client 2 registers the CQ listener with where name = emmanuel. The filter knowing for which client it filters, it will be able to only return the keys that match the query. This all sounds a bit like remote code exectution to me? You're asking for the client to pass some kind of executable thing that acts as a filter. That's a separate feature IMO, which I believe @Tristan is looking into. Once that's in place, I'm happy to enhance stuff in the remote event side to support it. I don't think you are correct. This is not remote execution in the sense of arbitrary code driven by the client. Remote execution will likely be triggered, render a result and stop. It will not send matching events in a continuous fashion. Plus remote execution will likely involve dynamic languages and I'm not sure we want to go that route for things like continuous query. To be clear, this is exactly the same as the filter parameters that Radim was asking for, right? From Infinispan's point of view, the filter just takes a String parameter, and the fact that that string can be parsed by the filter in a particular language is irrelevant. Not sure what string you are talking about. The filterid? ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
