Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
Thanks a lot Peter! I will make corrections suggested and issue a pull request! Will see this one through all the way till infinispan-cdi hooks into spi api we made! Cheers On 12-01-04 11:44 AM, Pete Muir wrote: Done, comments on the commit, sorry for the delay! On 28 Dec 2011, at 16:23, Vladimir Blagojevic wrote: Pete, Would you have a quick look at https://github.com/vblagoje/infinispan/compare/master...t_1636_m as I think this is what needs to be done! Still a bit rough phase but I would like to check with you before proceeding further! Regards, Vladimir On 11-12-23 12:18 PM, Pete Muir wrote: Yep. This is what CDI calls an injection target (an object which is the target for injection). So, what i would suggest you do is add an SPI to the map/reduce code that has callbacks which can be hooked into at the appropriate point by CDI or something else. So you would want a callback for inject(), postConstruct() and preDestroy(). You would provide a default impl for plain old java, which were basically no-ops, and then use e.g. the service loader to load other implementations, and call back to them. You then call this SPI at the relevant points in your code. Does that make any more sense? ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
On 20 Dec 2011, at 17:15, Vladimir Blagojevic wrote: On 11-12-20 1:54 PM, Pete Muir wrote: That should be pretty trivial. Only hard bit is integrating this into CDI module. Vladimir, do you provide a SPI for object instantiation and injection in the Map Reduce/Distributed Executor stuff? If so, Kevin can probably knock it up in an hour or two + tests ;-) No, I do not. I will talk to Kevin so he can let me know what he needs and then we can take it from there! Here is the SPI that CDI has for this purpose: http://docs.oracle.com/javaee/6/api/javax/enterprise/inject/spi/InjectionTarget.html I would suggest modelling after it, as it's proved successful, and will make CDI integration easy. (BTW I don't mean use it directly, I mean copy and paste it into infinispan code). Furthermore, what I would suggest you do is support the decorator pattern here, by creating one of these internally, and then allowing it to be got hold of by an extension, wrapped, and handed back to you. This makes it trivial for the extension to override the bits it wants. We also need to consider whether we want to allow overriding the construction and destruction of the object, as I guess by default you serialise/deserialize these? Might be useful to allow overriding anyway, in a fully stateless model (where changes are written back to the cache). ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
Pete, a bit more guidance please :-) I read http://docs.jboss.org/weld/reference/latest/en-US/html/extend.html and I looked at the infinispan-cdi code and how InfinispanExtension class wires into CDI container. I understand that part, well, sort of! I've seen how a class whose instances are supposed to be injected (DefaultCacheManager, AdvancedCache) is prepared for CDI in our infinispan-cdi module. Here is the SPI that CDI has for this purpose: http://docs.oracle.com/javaee/6/api/javax/enterprise/inject/spi/InjectionTarget.html I would suggest modelling after it, as it's proved successful, and will make CDI integration easy. (BTW I don't mean use it directly, I mean copy and paste it into infinispan code). I do not understand this part. What do you mean by copy and paste this into Infinispan code? Making sure we are on the same page: instances of Mapper (or Reducer) do not need to be injected somewhere; they are provided by the user and used in conjunction with MapReduceTask. Implementations of Mapper/Reducer interface need to have its field injected, once they are deseralized to remote Infinispan nodes, injected with appropriate Cache instance for example. Thanks, Vladimir Furthermore, what I would suggest you do is support the decorator pattern here, by creating one of these internally, and then allowing it to be got hold of by an extension, wrapped, and handed back to you. This makes it trivial for the extension to override the bits it wants. We also need to consider whether we want to allow overriding the construction and destruction of the object, as I guess by default you serialise/deserialize these? Might be useful to allow overriding anyway, in a fully stateless model (where changes are written back to the cache). ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
On 20 December 2011 13:35, Manik Surtani ma...@jboss.org wrote: What else do you think could/should be provided? Whatever people want to do with their data. Imagine you have to update all values by using some external service which provides some needed input, or you want to find all entries by some criteria and send them over to S3 via another service, or are storing movies and want to encode them using some other tool you have - these services could be invoked each time but assuming it has some expensive initialization it's likely one wants to inject references to initialized services and pools of resources. This user is asking for a reference for the Cache, that's the first step, but the next step is obviously to inject anything out of our control; recommending a new JNDI context + lookup per entry is likely not the most usable approach, while CDI should be simple to integrate and provide lot of value. On 19 Dec 2011, at 18:57, Sanne Grinovero wrote: Highlighting this use case: http://community.jboss.org/message/642300#642300 As I replied on the forum, I think we might need to improve the DistributedExecutorService by providing access to some context to the Future before it's executed remotely, as having a reference to the Cache it's being executed on seems very useful, but it might not be limited to the Cache only. I guess to make good use of it, someone might need to access just anything: CDI looks like a good fit? ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
So you're suggesting that the task is registered as a CDI bean on each remote node, so that requested services can be injected? Makes sense. But this would have to be a part of infinispan-cdi, as an extension to what we have in DistExec. I presume this is what you had in mind? On 20 Dec 2011, at 12:43, Sanne Grinovero wrote: On 20 December 2011 13:35, Manik Surtani ma...@jboss.org wrote: What else do you think could/should be provided? Whatever people want to do with their data. Imagine you have to update all values by using some external service which provides some needed input, or you want to find all entries by some criteria and send them over to S3 via another service, or are storing movies and want to encode them using some other tool you have - these services could be invoked each time but assuming it has some expensive initialization it's likely one wants to inject references to initialized services and pools of resources. This user is asking for a reference for the Cache, that's the first step, but the next step is obviously to inject anything out of our control; recommending a new JNDI context + lookup per entry is likely not the most usable approach, while CDI should be simple to integrate and provide lot of value. On 19 Dec 2011, at 18:57, Sanne Grinovero wrote: Highlighting this use case: http://community.jboss.org/message/642300#642300 As I replied on the forum, I think we might need to improve the DistributedExecutorService by providing access to some context to the Future before it's executed remotely, as having a reference to the Cache it's being executed on seems very useful, but it might not be limited to the Cache only. I guess to make good use of it, someone might need to access just anything: CDI looks like a good fit? ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
On 20 December 2011 13:47, Manik Surtani ma...@jboss.org wrote: So you're suggesting that the task is registered as a CDI bean on each remote node, so that requested services can be injected? Makes sense. But this would have to be a part of infinispan-cdi, as an extension to what we have in DistExec. I presume this is what you had in mind? Yes. Anyway a first step would be to provide a reference to the Cache, both to DistExec and MapReduce tasks, this should be doable even without introducing infinispan-cdi; but we need a cool proposal in terms of API - especially if @Inject is not supported that makes it harder to design. Sanne On 20 Dec 2011, at 12:43, Sanne Grinovero wrote: On 20 December 2011 13:35, Manik Surtani ma...@jboss.org wrote: What else do you think could/should be provided? Whatever people want to do with their data. Imagine you have to update all values by using some external service which provides some needed input, or you want to find all entries by some criteria and send them over to S3 via another service, or are storing movies and want to encode them using some other tool you have - these services could be invoked each time but assuming it has some expensive initialization it's likely one wants to inject references to initialized services and pools of resources. This user is asking for a reference for the Cache, that's the first step, but the next step is obviously to inject anything out of our control; recommending a new JNDI context + lookup per entry is likely not the most usable approach, while CDI should be simple to integrate and provide lot of value. On 19 Dec 2011, at 18:57, Sanne Grinovero wrote: Highlighting this use case: http://community.jboss.org/message/642300#642300 As I replied on the forum, I think we might need to improve the DistributedExecutorService by providing access to some context to the Future before it's executed remotely, as having a reference to the Cache it's being executed on seems very useful, but it might not be limited to the Cache only. I guess to make good use of it, someone might need to access just anything: CDI looks like a good fit? ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
On 20 December 2011 14:07, Manik Surtani ma...@jboss.org wrote: Yup, I agree. Care to create JIRAs for these two? I think the first one (adding a cache reference to the task) should be for 5.1.0. For 5.1: https://issues.jboss.org/browse/ISPN-1634 For 6.0 (?) : https://issues.jboss.org/browse/ISPN-1636 ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
On 11-12-20 9:54 AM, Sanne Grinovero wrote: On 20 December 2011 13:47, Manik Surtanima...@jboss.org wrote: So you're suggesting that the task is registered as a CDI bean on each remote node, so that requested services can be injected? Makes sense. But this would have to be a part of infinispan-cdi, as an extension to what we have in DistExec. I presume this is what you had in mind? Yes. Anyway a first step would be to provide a reference to the Cache, both to DistExec and MapReduce tasks, this should be doable even without introducing infinispan-cdi; but we need a cool proposal in terms of API - especially if @Inject is not supported that makes it harder to design. Sanne Makes sense! Note that implementors of DistributedCallable task in distributed executors already receive Cache instance through API! We can have Mapper and Reducer receive owning cache instance through API as well! However, if it can be done through @Inject mechanism maybe that would be better! ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
Vladimir, for you? :) On 20 Dec 2011, at 13:21, Sanne Grinovero wrote: On 20 December 2011 14:07, Manik Surtani ma...@jboss.org wrote: Yup, I agree. Care to create JIRAs for these two? I think the first one (adding a cache reference to the task) should be for 5.1.0. For 5.1: https://issues.jboss.org/browse/ISPN-1634 For 6.0 (?) : https://issues.jboss.org/browse/ISPN-1636 ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
Sure, I'd like to hear Pete's opinion here. Can we inject pretty much anything from application scope or do we have to do something special in Mapper and Reducer to prepare objects that can be injected? I'd rather leave interfaces as they are and use injection mechanism to inject objects needed by implementors of Mapper and Reducer. Today it is cache, tomorrow people will ask for something else...and it seems to me injection mechanism is the way to go here! Pete? Vladimir On 11-12-20 10:55 AM, Manik Surtani wrote: Vladimir, for you? :) On 20 Dec 2011, at 13:21, Sanne Grinovero wrote: On 20 December 2011 14:07, Manik Surtanima...@jboss.org wrote: Yup, I agree. Care to create JIRAs for these two? I think the first one (adding a cache reference to the task) should be for 5.1.0. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
Are you asking what else needs to be added to the Mapper and Reducer interfaces to make them CDI-friendly so that implementations can have stuff injected? I suspect nothing need be done except that on the receiving node, if these implementations are annotated as CDI beans, then they should be passed to the CDI container for proper setup before being used. On 20 Dec 2011, at 14:00, Vladimir Blagojevic wrote: Sure, I'd like to hear Pete's opinion here. Can we inject pretty much anything from application scope or do we have to do something special in Mapper and Reducer to prepare objects that can be injected? I'd rather leave interfaces as they are and use injection mechanism to inject objects needed by implementors of Mapper and Reducer. Today it is cache, tomorrow people will ask for something else...and it seems to me injection mechanism is the way to go here! Pete? Vladimir On 11-12-20 10:55 AM, Manik Surtani wrote: Vladimir, for you? :) On 20 Dec 2011, at 13:21, Sanne Grinovero wrote: On 20 December 2011 14:07, Manik Surtanima...@jboss.org wrote: Yup, I agree. Care to create JIRAs for these two? I think the first one (adding a cache reference to the task) should be for 5.1.0. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
Yeah, that is my question. In conclusion, Mapper/Reducer instances should be passed to CDI container for resolution once they are migrated to execution nodes, but before they are executed on those nodes On 11-12-20 12:29 PM, Manik Surtani wrote: Are you asking what else needs to be added to the Mapper and Reducer interfaces to make them CDI-friendly so that implementations can have stuff injected? I suspect nothing need be done except that on the receiving node, if these implementations are annotated as CDI beans, then they should be passed to the CDI container for proper setup before being used. On 20 Dec 2011, at 14:00, Vladimir Blagojevic wrote: Sure, I'd like to hear Pete's opinion here. Can we inject pretty much anything from application scope or do we have to do something special in Mapper and Reducer to prepare objects that can be injected? I'd rather leave interfaces as they are and use injection mechanism to inject objects needed by implementors of Mapper and Reducer. Today it is cache, tomorrow people will ask for something else...and it seems to me injection mechanism is the way to go here! Pete? Vladimir On 11-12-20 10:55 AM, Manik Surtani wrote: Vladimir, for you? :) On 20 Dec 2011, at 13:21, Sanne Grinovero wrote: On 20 December 2011 14:07, Manik Surtanima...@jboss.org wrote: Yup, I agree. Care to create JIRAs for these two? I think the first one (adding a cache reference to the task) should be for 5.1.0. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Distributed Executors in write mode, need reference to Cache
That should be pretty trivial. Only hard bit is integrating this into CDI module. Vladimir, do you provide a SPI for object instantiation and injection in the Map Reduce/Distributed Executor stuff? If so, Kevin can probably knock it up in an hour or two + tests ;-) On 20 Dec 2011, at 16:47, Vladimir Blagojevic wrote: Yeah, that is my question. In conclusion, Mapper/Reducer instances should be passed to CDI container for resolution once they are migrated to execution nodes, but before they are executed on those nodes On 11-12-20 12:29 PM, Manik Surtani wrote: Are you asking what else needs to be added to the Mapper and Reducer interfaces to make them CDI-friendly so that implementations can have stuff injected? I suspect nothing need be done except that on the receiving node, if these implementations are annotated as CDI beans, then they should be passed to the CDI container for proper setup before being used. On 20 Dec 2011, at 14:00, Vladimir Blagojevic wrote: Sure, I'd like to hear Pete's opinion here. Can we inject pretty much anything from application scope or do we have to do something special in Mapper and Reducer to prepare objects that can be injected? I'd rather leave interfaces as they are and use injection mechanism to inject objects needed by implementors of Mapper and Reducer. Today it is cache, tomorrow people will ask for something else...and it seems to me injection mechanism is the way to go here! Pete? Vladimir On 11-12-20 10:55 AM, Manik Surtani wrote: Vladimir, for you? :) On 20 Dec 2011, at 13:21, Sanne Grinovero wrote: On 20 December 2011 14:07, Manik Surtanima...@jboss.org wrote: Yup, I agree. Care to create JIRAs for these two? I think the first one (adding a cache reference to the task) should be for 5.1.0. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] Distributed Executors in write mode, need reference to Cache
Highlighting this use case: http://community.jboss.org/message/642300#642300 As I replied on the forum, I think we might need to improve the DistributedExecutorService by providing access to some context to the Future before it's executed remotely, as having a reference to the Cache it's being executed on seems very useful, but it might not be limited to the Cache only. I guess to make good use of it, someone might need to access just anything: CDI looks like a good fit? ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
