Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
Just a heads up: I have not forgotten about this. In the pull req I sent last night, I've gone for the interface option, but I like the concrete class option too (particularly from the POV of being able to copy it and pass it on). However, work still needs to be done to pass the full metadata object down to the container in order to store it all, which will be done when REST compatibility is added. So, I'm delaying the decision on whether to go with interface vs concrete class, until I have worked on the use cases and I can play with both options, and see how extensible the concrete class path can be. Cheers, On Apr 8, 2013, at 6:51 PM, Galder Zamarreño wrote: > > On Apr 8, 2013, at 4:09 PM, Manik Surtani wrote: > >> Tombstones as well as external versioning - something Hibernate 2LC has >> needed for a while (and Max doesn't ever stop bugging me about!) >> >> Re: the serialisability, how about this: why make Metadata in interface? >> Why not a concrete class, with a fixed set of attributes (lifespan, maxIdle, >> Version (interface), etc). Then we ship an externalizer for this Metadata >> class. > > ^ That's certainly an option, but it's gotta be extensible (and retrievable), > so that server's can build on top of it. For example, REST server might wanna > add MIME info on top of it. It's got to be able to extend Metadata concrete > class, so that it can be passed in (and of course, be able to retrieve it > back), and this is more akward with concrete classes as opposed to interfaces. > >> >> - M >> >> On 8 Apr 2013, at 13:52, Sanne Grinovero wrote: >> >>> Got it, thanks! >>> +1 especially as it helps bringing tombstones, an urgent feature IMHO. >>> >>> Sanne >>> >>> On 8 April 2013 13:11, Galder Zamarreño wrote: On Apr 8, 2013, at 1:46 PM, Sanne Grinovero wrote: > I fail to understand the purpose of the feature then. What prevents me > to use the existing code today just storing some extra fields in my > custom values? ^ Nothing, this is doable. > What do we get by adding this code? ^ You avoid the need of the wrapper since we already have a wrapper internally, which is ICE. ICEs can already keep versions around, why do I need a wrapper class that stores a version for Hot Rod server? Down the line, we could decide to leave the metadata around to better support use cases like this: https://issues.jboss.org/browse/ISPN-506 https://community.jboss.org/wiki/VersioningDesignDocument - The tombstones that Max refers to could potentially be tombstone ICEs with only metadata info. Cheers, > > Sanne > > On 8 April 2013 12:40, Galder Zamarreño wrote: >> >> On Apr 8, 2013, at 1:26 PM, Sanne Grinovero wrote: >> >>> On 8 April 2013 12:06, Galder Zamarreño wrote: On Apr 8, 2013, at 12:56 PM, Sanne Grinovero wrote: > > > > On 8 April 2013 11:44, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:35 PM, Galder Zamarreño > wrote: > >> >> On Apr 8, 2013, at 11:17 AM, Manik Surtani >> wrote: >> >>> All sounds very good. One important thing to consider is that the >>> reference to Metadata passed in by the client app will be tied to >>> the ICE for the entire lifespan of the ICE. You'll need to think >>> about a defensive copy or some other form of making the Metadata >>> immutable (by the user application, at least) the moment it is >>> passed in. >> >> ^ Excellent point, it could be a nightmare if users could change the >> metadata reference by the ICE at will. I'll have a think on how to >> best achieve this. > > ^ The metadata is gonna have to be marshalled somehow to ship to > other nodes, so that could be a way to achieve it, by enforcing this > somehow. When the cache receives it, it can marshaller/unmarshall it > to make a copy > > One way would be to make Metadata extend Serializable, but not keen > on that. Another would be to somehow force the interface to define > the Externalizer to use (i.e. an interface method like > getExternalizer()), but that's akward when it comes to unmarshalling… > what about forcing the Metadata object to be provided with a > @SerializeWith annotation? > > Why is getExternalizer() awkward for unmarshalling? ^ Because you don't have an instance yet, so what's the Externalizer for it? IOW, there's no much point to doing that, simply register it depending on your desire: https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >>> >>> That's w
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 6:59 PM, Tristan Tarrant wrote: > > > On 04/08/2013 06:51 PM, Galder Zamarreño wrote: >> ^ That's certainly an option, but it's gotta be extensible (and >> retrievable), so that server's can build on top of it. For example, >> REST server might wanna add MIME info on top of it. It's got to be >> able to extend Metadata concrete class, so that it can be passed in >> (and of course, be able to retrieve it back), and this is more akward >> with concrete classes as opposed to interfaces. > Actually I'd like MIME info for HotRod too (or maybe just a way to > deduct it from the data-type). The Hot Rod protocol does not allow MIME to be sent it (yet), so the only way you'll be able to get MIME info is if either this is stored via embedded cache or REST endpoints, in which case, this info will need to be encoded into a byte[] somehow to be retrievable by Hot Rod, which will be something to be handled by the type converter… I haven't got a solution yet for this, but it'll be the next step. > > Tristan > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño gal...@redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On 04/08/2013 06:51 PM, Galder Zamarreño wrote: > ^ That's certainly an option, but it's gotta be extensible (and > retrievable), so that server's can build on top of it. For example, > REST server might wanna add MIME info on top of it. It's got to be > able to extend Metadata concrete class, so that it can be passed in > (and of course, be able to retrieve it back), and this is more akward > with concrete classes as opposed to interfaces. Actually I'd like MIME info for HotRod too (or maybe just a way to deduct it from the data-type). Tristan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 4:09 PM, Manik Surtani wrote: > Tombstones as well as external versioning - something Hibernate 2LC has > needed for a while (and Max doesn't ever stop bugging me about!) > > Re: the serialisability, how about this: why make Metadata in interface? Why > not a concrete class, with a fixed set of attributes (lifespan, maxIdle, > Version (interface), etc). Then we ship an externalizer for this Metadata > class. ^ That's certainly an option, but it's gotta be extensible (and retrievable), so that server's can build on top of it. For example, REST server might wanna add MIME info on top of it. It's got to be able to extend Metadata concrete class, so that it can be passed in (and of course, be able to retrieve it back), and this is more akward with concrete classes as opposed to interfaces. > > - M > > On 8 Apr 2013, at 13:52, Sanne Grinovero wrote: > >> Got it, thanks! >> +1 especially as it helps bringing tombstones, an urgent feature IMHO. >> >> Sanne >> >> On 8 April 2013 13:11, Galder Zamarreño wrote: >>> >>> On Apr 8, 2013, at 1:46 PM, Sanne Grinovero wrote: >>> I fail to understand the purpose of the feature then. What prevents me to use the existing code today just storing some extra fields in my custom values? >>> >>> ^ Nothing, this is doable. >>> What do we get by adding this code? >>> >>> ^ You avoid the need of the wrapper since we already have a wrapper >>> internally, which is ICE. ICEs can already keep versions around, why do I >>> need a wrapper class that stores a version for Hot Rod server? >>> >>> Down the line, we could decide to leave the metadata around to better >>> support use cases like this: >>> https://issues.jboss.org/browse/ISPN-506 >>> https://community.jboss.org/wiki/VersioningDesignDocument - The tombstones >>> that Max refers to could potentially be tombstone ICEs with only metadata >>> info. >>> >>> Cheers, >>> Sanne On 8 April 2013 12:40, Galder Zamarreño wrote: > > On Apr 8, 2013, at 1:26 PM, Sanne Grinovero wrote: > >> On 8 April 2013 12:06, Galder Zamarreño wrote: >>> >>> On Apr 8, 2013, at 12:56 PM, Sanne Grinovero >>> wrote: >>> On 8 April 2013 11:44, Galder Zamarreño wrote: On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > On Apr 8, 2013, at 11:17 AM, Manik Surtani > wrote: > >> All sounds very good. One important thing to consider is that the >> reference to Metadata passed in by the client app will be tied to >> the ICE for the entire lifespan of the ICE. You'll need to think >> about a defensive copy or some other form of making the Metadata >> immutable (by the user application, at least) the moment it is >> passed in. > > ^ Excellent point, it could be a nightmare if users could change the > metadata reference by the ICE at will. I'll have a think on how to > best achieve this. ^ The metadata is gonna have to be marshalled somehow to ship to other nodes, so that could be a way to achieve it, by enforcing this somehow. When the cache receives it, it can marshaller/unmarshall it to make a copy One way would be to make Metadata extend Serializable, but not keen on that. Another would be to somehow force the interface to define the Externalizer to use (i.e. an interface method like getExternalizer()), but that's akward when it comes to unmarshalling… what about forcing the Metadata object to be provided with a @SerializeWith annotation? Why is getExternalizer() awkward for unmarshalling? >>> >>> ^ Because you don't have an instance yet, so what's the Externalizer >>> for it? IOW, there's no much point to doing that, simply register it >>> depending on your desire: >>> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >> >> That's what I would expect. >> >>> I would expect you to have the marshaller already known during deserialization. >>> >>> You would, as long as you follow the instructions in >>> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >>> Agreed that extensing Serializable is not a good idea. Are you thinking about the impact on CacheStore(s) and state transfer? >>> >>> ^ What about it in particular? >>> Eviction of no longer used metadata ? >>> >>> ^ Since the metadata is part of the entry, it'd initially go when the >>> entry is evicted. We might wanna leave it around in some cases… but >>> it'd be for other use cases. >> >> I thought the p
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
Tombstones as well as external versioning - something Hibernate 2LC has needed for a while (and Max doesn't ever stop bugging me about!) Re: the serialisability, how about this: why make Metadata in interface? Why not a concrete class, with a fixed set of attributes (lifespan, maxIdle, Version (interface), etc). Then we ship an externalizer for this Metadata class. - M On 8 Apr 2013, at 13:52, Sanne Grinovero wrote: > Got it, thanks! > +1 especially as it helps bringing tombstones, an urgent feature IMHO. > > Sanne > > On 8 April 2013 13:11, Galder Zamarreño wrote: >> >> On Apr 8, 2013, at 1:46 PM, Sanne Grinovero wrote: >> >>> I fail to understand the purpose of the feature then. What prevents me >>> to use the existing code today just storing some extra fields in my >>> custom values? >> >> ^ Nothing, this is doable. >> >>> What do we get by adding this code? >> >> ^ You avoid the need of the wrapper since we already have a wrapper >> internally, which is ICE. ICEs can already keep versions around, why do I >> need a wrapper class that stores a version for Hot Rod server? >> >> Down the line, we could decide to leave the metadata around to better >> support use cases like this: >> https://issues.jboss.org/browse/ISPN-506 >> https://community.jboss.org/wiki/VersioningDesignDocument - The tombstones >> that Max refers to could potentially be tombstone ICEs with only metadata >> info. >> >> Cheers, >> >>> >>> Sanne >>> >>> On 8 April 2013 12:40, Galder Zamarreño wrote: On Apr 8, 2013, at 1:26 PM, Sanne Grinovero wrote: > On 8 April 2013 12:06, Galder Zamarreño wrote: >> >> On Apr 8, 2013, at 12:56 PM, Sanne Grinovero >> wrote: >> >>> >>> >>> >>> On 8 April 2013 11:44, Galder Zamarreño wrote: >>> >>> On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: >>> On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > All sounds very good. One important thing to consider is that the > reference to Metadata passed in by the client app will be tied to the > ICE for the entire lifespan of the ICE. You'll need to think about a > defensive copy or some other form of making the Metadata immutable > (by the user application, at least) the moment it is passed in. ^ Excellent point, it could be a nightmare if users could change the metadata reference by the ICE at will. I'll have a think on how to best achieve this. >>> >>> ^ The metadata is gonna have to be marshalled somehow to ship to other >>> nodes, so that could be a way to achieve it, by enforcing this somehow. >>> When the cache receives it, it can marshaller/unmarshall it to make a >>> copy >>> >>> One way would be to make Metadata extend Serializable, but not keen on >>> that. Another would be to somehow force the interface to define the >>> Externalizer to use (i.e. an interface method like getExternalizer()), >>> but that's akward when it comes to unmarshalling… what about forcing >>> the Metadata object to be provided with a @SerializeWith annotation? >>> >>> Why is getExternalizer() awkward for unmarshalling? >> >> ^ Because you don't have an instance yet, so what's the Externalizer for >> it? IOW, there's no much point to doing that, simply register it >> depending on your desire: >> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers > > That's what I would expect. > >> >>> I would expect you to have the marshaller already known during >>> deserialization. >> >> You would, as long as you follow the instructions in >> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >> >>> Agreed that extensing Serializable is not a good idea. >>> >>> Are you thinking about the impact on CacheStore(s) and state transfer? >> >> ^ What about it in particular? >> >>> Eviction of no longer used metadata ? >> >> ^ Since the metadata is part of the entry, it'd initially go when the >> entry is evicted. We might wanna leave it around in some cases… but it'd >> be for other use cases. > > I thought the plan was to have entries refer to the metadata, but that > different entries sharing the same metadata would point to the same > instance. ^ Could be, but most likely not. > So this metadata needs to be stored separately in the CacheStore, > preloaded as appropriate, transferred during state transfer, > passivated when convenient and cleaned up when no longer referred to. ^ Well, it's part of the internal cache entry, so it'd be treated just like ICE. > Am I wrong? Seems you plan to store a copy of the metadata within each > ICE
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
Got it, thanks! +1 especially as it helps bringing tombstones, an urgent feature IMHO. Sanne On 8 April 2013 13:11, Galder Zamarreño wrote: > > On Apr 8, 2013, at 1:46 PM, Sanne Grinovero wrote: > >> I fail to understand the purpose of the feature then. What prevents me >> to use the existing code today just storing some extra fields in my >> custom values? > > ^ Nothing, this is doable. > >> What do we get by adding this code? > > ^ You avoid the need of the wrapper since we already have a wrapper > internally, which is ICE. ICEs can already keep versions around, why do I > need a wrapper class that stores a version for Hot Rod server? > > Down the line, we could decide to leave the metadata around to better support > use cases like this: > https://issues.jboss.org/browse/ISPN-506 > https://community.jboss.org/wiki/VersioningDesignDocument - The tombstones > that Max refers to could potentially be tombstone ICEs with only metadata > info. > > Cheers, > >> >> Sanne >> >> On 8 April 2013 12:40, Galder Zamarreño wrote: >>> >>> On Apr 8, 2013, at 1:26 PM, Sanne Grinovero wrote: >>> On 8 April 2013 12:06, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:56 PM, Sanne Grinovero wrote: > >> >> >> >> On 8 April 2013 11:44, Galder Zamarreño wrote: >> >> On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: >> >>> >>> On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: >>> All sounds very good. One important thing to consider is that the reference to Metadata passed in by the client app will be tied to the ICE for the entire lifespan of the ICE. You'll need to think about a defensive copy or some other form of making the Metadata immutable (by the user application, at least) the moment it is passed in. >>> >>> ^ Excellent point, it could be a nightmare if users could change the >>> metadata reference by the ICE at will. I'll have a think on how to best >>> achieve this. >> >> ^ The metadata is gonna have to be marshalled somehow to ship to other >> nodes, so that could be a way to achieve it, by enforcing this somehow. >> When the cache receives it, it can marshaller/unmarshall it to make a >> copy >> >> One way would be to make Metadata extend Serializable, but not keen on >> that. Another would be to somehow force the interface to define the >> Externalizer to use (i.e. an interface method like getExternalizer()), >> but that's akward when it comes to unmarshalling… what about forcing the >> Metadata object to be provided with a @SerializeWith annotation? >> >> Why is getExternalizer() awkward for unmarshalling? > > ^ Because you don't have an instance yet, so what's the Externalizer for > it? IOW, there's no much point to doing that, simply register it > depending on your desire: > https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers That's what I would expect. > >> I would expect you to have the marshaller already known during >> deserialization. > > You would, as long as you follow the instructions in > https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers > >> Agreed that extensing Serializable is not a good idea. >> >> Are you thinking about the impact on CacheStore(s) and state transfer? > > ^ What about it in particular? > >> Eviction of no longer used metadata ? > > ^ Since the metadata is part of the entry, it'd initially go when the > entry is evicted. We might wanna leave it around in some cases… but it'd > be for other use cases. I thought the plan was to have entries refer to the metadata, but that different entries sharing the same metadata would point to the same instance. >>> >>> ^ Could be, but most likely not. >>> So this metadata needs to be stored separately in the CacheStore, preloaded as appropriate, transferred during state transfer, passivated when convenient and cleaned up when no longer referred to. >>> >>> ^ Well, it's part of the internal cache entry, so it'd be treated just like >>> ICE. >>> Am I wrong? Seems you plan to store a copy of the metadata within each ICE. >>> >>> ^ The idea is to store it alongside right now, but maybe at some point it >>> might make sense to leave it around (i.e. for 2LC use case), but this won't >>> be done yet. >>> > > I'm also considering separating the serialization/marshalling concerns > from the defensive copying concerns. IOW, add a copy() method to the > Metadata interface, or have a separate interface for those externally > provided objects that require to be defensive copied. IOW, do something > like what Scala Case Classes do with their copy() method, but without the > issues
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 1:46 PM, Sanne Grinovero wrote: > I fail to understand the purpose of the feature then. What prevents me > to use the existing code today just storing some extra fields in my > custom values? ^ Nothing, this is doable. > What do we get by adding this code? ^ You avoid the need of the wrapper since we already have a wrapper internally, which is ICE. ICEs can already keep versions around, why do I need a wrapper class that stores a version for Hot Rod server? Down the line, we could decide to leave the metadata around to better support use cases like this: https://issues.jboss.org/browse/ISPN-506 https://community.jboss.org/wiki/VersioningDesignDocument - The tombstones that Max refers to could potentially be tombstone ICEs with only metadata info. Cheers, > > Sanne > > On 8 April 2013 12:40, Galder Zamarreño wrote: >> >> On Apr 8, 2013, at 1:26 PM, Sanne Grinovero wrote: >> >>> On 8 April 2013 12:06, Galder Zamarreño wrote: On Apr 8, 2013, at 12:56 PM, Sanne Grinovero wrote: > > > > On 8 April 2013 11:44, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > >> >> On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: >> >>> All sounds very good. One important thing to consider is that the >>> reference to Metadata passed in by the client app will be tied to the >>> ICE for the entire lifespan of the ICE. You'll need to think about a >>> defensive copy or some other form of making the Metadata immutable (by >>> the user application, at least) the moment it is passed in. >> >> ^ Excellent point, it could be a nightmare if users could change the >> metadata reference by the ICE at will. I'll have a think on how to best >> achieve this. > > ^ The metadata is gonna have to be marshalled somehow to ship to other > nodes, so that could be a way to achieve it, by enforcing this somehow. > When the cache receives it, it can marshaller/unmarshall it to make a copy > > One way would be to make Metadata extend Serializable, but not keen on > that. Another would be to somehow force the interface to define the > Externalizer to use (i.e. an interface method like getExternalizer()), > but that's akward when it comes to unmarshalling… what about forcing the > Metadata object to be provided with a @SerializeWith annotation? > > Why is getExternalizer() awkward for unmarshalling? ^ Because you don't have an instance yet, so what's the Externalizer for it? IOW, there's no much point to doing that, simply register it depending on your desire: https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >>> >>> That's what I would expect. >>> > I would expect you to have the marshaller already known during > deserialization. You would, as long as you follow the instructions in https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers > Agreed that extensing Serializable is not a good idea. > > Are you thinking about the impact on CacheStore(s) and state transfer? ^ What about it in particular? > Eviction of no longer used metadata ? ^ Since the metadata is part of the entry, it'd initially go when the entry is evicted. We might wanna leave it around in some cases… but it'd be for other use cases. >>> >>> I thought the plan was to have entries refer to the metadata, but that >>> different entries sharing the same metadata would point to the same >>> instance. >> >> ^ Could be, but most likely not. >> >>> So this metadata needs to be stored separately in the CacheStore, >>> preloaded as appropriate, transferred during state transfer, >>> passivated when convenient and cleaned up when no longer referred to. >> >> ^ Well, it's part of the internal cache entry, so it'd be treated just like >> ICE. >> >>> Am I wrong? Seems you plan to store a copy of the metadata within each ICE. >> >> ^ The idea is to store it alongside right now, but maybe at some point it >> might make sense to leave it around (i.e. for 2LC use case), but this won't >> be done yet. >> >>> >>> I'm also considering separating the serialization/marshalling concerns from the defensive copying concerns. IOW, add a copy() method to the Metadata interface, or have a separate interface for those externally provided objects that require to be defensive copied. IOW, do something like what Scala Case Classes do with their copy() method, but without the issues of clone… I need to investigate this further to come up with a nice solution. One positive side to splitting both concerns is speed. A Metadata implementation might have ways to make a copy of itself which are more eff
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Mon, Apr 8, 2013 at 2:36 PM, Galder Zamarreño wrote: > > On Apr 8, 2013, at 1:11 PM, Dan Berindei wrote: > > > > > > > > > On Mon, Apr 8, 2013 at 1:44 PM, Galder Zamarreño > wrote: > > > > On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > > > > > > > On Apr 8, 2013, at 11:17 AM, Manik Surtani > wrote: > > > > > >> All sounds very good. One important thing to consider is that the > reference to Metadata passed in by the client app will be tied to the ICE > for the entire lifespan of the ICE. You'll need to think about a defensive > copy or some other form of making the Metadata immutable (by the user > application, at least) the moment it is passed in. > > > > > > ^ Excellent point, it could be a nightmare if users could change the > metadata reference by the ICE at will. I'll have a think on how to best > achieve this. > > > > ^ The metadata is gonna have to be marshalled somehow to ship to other > nodes, so that could be a way to achieve it, by enforcing this somehow. > When the cache receives it, it can marshaller/unmarshall it to make a copy > > > > > > If Metadata is just an interface, nothing is stopping the user from > implementing maxIdle() to return Random.maxLong(). Besides, local caches > need to support Metadata as well, and we shouldn't force > serialization/deserialization for local caches. > > > > So I think we'd be better off documenting that Metadata objects should > not change after they are inserted in the cache, just like keys and values. > > > > > > One way would be to make Metadata extend Serializable, but not keen on > that. Another would be to somehow force the interface to define the > Externalizer to use (i.e. an interface method like getExternalizer()), but > that's akward when it comes to unmarshalling… what about forcing the > Metadata object to be provided with a @SerializeWith annotation? > > > > Any other ideas? > > > > > > Why force anything? I think Metadata instances should be treated just > like keys and values, so they should be able to use Externalizers (via > @SerializeWith), Serializable, or Externalizable, depending on the user's > requirements. > > ^ I agree. > > What do you think of my suggestion in the other email to separate both > concerns and somehow enforce a copy of the object to be provided instead? > > I wrote my reply before I saw your other email :) Having said that, I still think enforcing a copy doesn't make sense (see my other comment). > > > > > > > > > > Cheers, > > > > > >> > > >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > > >> > > >>> Hi all, > > >>> > > >>> As mentioned in > http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, > in paralell to the switch to Equivalent* collections, I was also working on > being able to pass metadata into Infinispan caches. This is done to better > support the ability to store custom metadata in Infinispan without the need > of extra wrappers. So, the idea is that InternalCacheEntry instances will > have a a reference to this Metadata. > > >>> > > >>> One of that metadata is version, which I've been using as test bed > to see if clients could pass succesfully version information via metadata. > As you already know, Hot Rod requires to store version information. Before, > this was stored in a class called CacheValue alongside the value itself, > but the work I've done in [1], this is passed via the new API I've added in > [2]. > > >>> > > >>> So, I'd like to get some thoughts on this new API. I hope that with > these new put/replace versions, we can get rid of the nightmare which is > all the other put/replace calls taking lifespan and/or maxIdle information. > In the end, I think there should be two basic puts: > > >>> > > >>> - put(K, V) > > >>> - put(K, V, Metadata) > > >>> > > >>> And their equivalents. > > >>> > > >>> IMPORTANT NOTE 1: The implementation details are bound to change, > because the entire Metadata needs to be stored in InternalCacheEntry, not > just version, lifespan..etc. I'll further develop the implementation once I > get into adding more metadata, i.e. when working on interoperability with > REST. So, don't pay too much attention to the implementation itself, focus > on the AdvancedCache API itself and let's refine that. > > >>> > > >>> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, > so please let's avoid discussing it in this email thread. Once I have a > more final version I'll send an email about it. > > >>> > > >>> Apart from working on enhancements to the API, I'm now carry on > tackling the interoperability work with aim to have an initial version of > the Embedded <-> Hot Rod interoperability as first step. Once that's in, it > can be released to get early feedback while the rest of interoperability > modes are developed. > > >>> > > >>> Cheers, > > >>> > > >>> [1] > https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d > > >>> [2] > https://github.com/galderz/infinispan/commit/a359
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
I fail to understand the purpose of the feature then. What prevents me to use the existing code today just storing some extra fields in my custom values? What do we get by adding this code? Sanne On 8 April 2013 12:40, Galder Zamarreño wrote: > > On Apr 8, 2013, at 1:26 PM, Sanne Grinovero wrote: > >> On 8 April 2013 12:06, Galder Zamarreño wrote: >>> >>> On Apr 8, 2013, at 12:56 PM, Sanne Grinovero wrote: >>> On 8 April 2013 11:44, Galder Zamarreño wrote: On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > >> All sounds very good. One important thing to consider is that the >> reference to Metadata passed in by the client app will be tied to the >> ICE for the entire lifespan of the ICE. You'll need to think about a >> defensive copy or some other form of making the Metadata immutable (by >> the user application, at least) the moment it is passed in. > > ^ Excellent point, it could be a nightmare if users could change the > metadata reference by the ICE at will. I'll have a think on how to best > achieve this. ^ The metadata is gonna have to be marshalled somehow to ship to other nodes, so that could be a way to achieve it, by enforcing this somehow. When the cache receives it, it can marshaller/unmarshall it to make a copy One way would be to make Metadata extend Serializable, but not keen on that. Another would be to somehow force the interface to define the Externalizer to use (i.e. an interface method like getExternalizer()), but that's akward when it comes to unmarshalling… what about forcing the Metadata object to be provided with a @SerializeWith annotation? Why is getExternalizer() awkward for unmarshalling? >>> >>> ^ Because you don't have an instance yet, so what's the Externalizer for >>> it? IOW, there's no much point to doing that, simply register it depending >>> on your desire: >>> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >> >> That's what I would expect. >> >>> I would expect you to have the marshaller already known during deserialization. >>> >>> You would, as long as you follow the instructions in >>> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >>> Agreed that extensing Serializable is not a good idea. Are you thinking about the impact on CacheStore(s) and state transfer? >>> >>> ^ What about it in particular? >>> Eviction of no longer used metadata ? >>> >>> ^ Since the metadata is part of the entry, it'd initially go when the entry >>> is evicted. We might wanna leave it around in some cases… but it'd be for >>> other use cases. >> >> I thought the plan was to have entries refer to the metadata, but that >> different entries sharing the same metadata would point to the same >> instance. > > ^ Could be, but most likely not. > >> So this metadata needs to be stored separately in the CacheStore, >> preloaded as appropriate, transferred during state transfer, >> passivated when convenient and cleaned up when no longer referred to. > > ^ Well, it's part of the internal cache entry, so it'd be treated just like > ICE. > >> Am I wrong? Seems you plan to store a copy of the metadata within each ICE. > > ^ The idea is to store it alongside right now, but maybe at some point it > might make sense to leave it around (i.e. for 2LC use case), but this won't > be done yet. > >> >> >>> >>> I'm also considering separating the serialization/marshalling concerns from >>> the defensive copying concerns. IOW, add a copy() method to the Metadata >>> interface, or have a separate interface for those externally provided >>> objects that require to be defensive copied. IOW, do something like what >>> Scala Case Classes do with their copy() method, but without the issues of >>> clone… I need to investigate this further to come up with a nice solution. >>> >>> One positive side to splitting both concerns is speed. A Metadata >>> implementation might have ways to make a copy of itself which are more >>> efficient than marshalling/unmarshalling. >>> >>> Thoughts? >>> Sanne Any other ideas? > > Cheers, > >> >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: >> >>> Hi all, >>> >>> As mentioned in >>> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, >>> in paralell to the switch to Equivalent* collections, I was also >>> working on being able to pass metadata into Infinispan caches. This is >>> done to better support the ability to store custom metadata in >>> Infinispan without the need of extra wrappers. So, the idea is that >>> InternalCacheEntry instances will have a a reference to this Metadata. >>> >>> One of that met
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 1:26 PM, Sanne Grinovero wrote: > On 8 April 2013 12:06, Galder Zamarreño wrote: >> >> On Apr 8, 2013, at 12:56 PM, Sanne Grinovero wrote: >> >>> >>> >>> >>> On 8 April 2013 11:44, Galder Zamarreño wrote: >>> >>> On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: >>> On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > All sounds very good. One important thing to consider is that the > reference to Metadata passed in by the client app will be tied to the ICE > for the entire lifespan of the ICE. You'll need to think about a > defensive copy or some other form of making the Metadata immutable (by > the user application, at least) the moment it is passed in. ^ Excellent point, it could be a nightmare if users could change the metadata reference by the ICE at will. I'll have a think on how to best achieve this. >>> >>> ^ The metadata is gonna have to be marshalled somehow to ship to other >>> nodes, so that could be a way to achieve it, by enforcing this somehow. >>> When the cache receives it, it can marshaller/unmarshall it to make a copy >>> >>> One way would be to make Metadata extend Serializable, but not keen on >>> that. Another would be to somehow force the interface to define the >>> Externalizer to use (i.e. an interface method like getExternalizer()), but >>> that's akward when it comes to unmarshalling… what about forcing the >>> Metadata object to be provided with a @SerializeWith annotation? >>> >>> Why is getExternalizer() awkward for unmarshalling? >> >> ^ Because you don't have an instance yet, so what's the Externalizer for it? >> IOW, there's no much point to doing that, simply register it depending on >> your desire: >> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers > > That's what I would expect. > >> >>> I would expect you to have the marshaller already known during >>> deserialization. >> >> You would, as long as you follow the instructions in >> https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers >> >>> Agreed that extensing Serializable is not a good idea. >>> >>> Are you thinking about the impact on CacheStore(s) and state transfer? >> >> ^ What about it in particular? >> >>> Eviction of no longer used metadata ? >> >> ^ Since the metadata is part of the entry, it'd initially go when the entry >> is evicted. We might wanna leave it around in some cases… but it'd be for >> other use cases. > > I thought the plan was to have entries refer to the metadata, but that > different entries sharing the same metadata would point to the same > instance. ^ Could be, but most likely not. > So this metadata needs to be stored separately in the CacheStore, > preloaded as appropriate, transferred during state transfer, > passivated when convenient and cleaned up when no longer referred to. ^ Well, it's part of the internal cache entry, so it'd be treated just like ICE. > Am I wrong? Seems you plan to store a copy of the metadata within each ICE. ^ The idea is to store it alongside right now, but maybe at some point it might make sense to leave it around (i.e. for 2LC use case), but this won't be done yet. > > >> >> I'm also considering separating the serialization/marshalling concerns from >> the defensive copying concerns. IOW, add a copy() method to the Metadata >> interface, or have a separate interface for those externally provided >> objects that require to be defensive copied. IOW, do something like what >> Scala Case Classes do with their copy() method, but without the issues of >> clone… I need to investigate this further to come up with a nice solution. >> >> One positive side to splitting both concerns is speed. A Metadata >> implementation might have ways to make a copy of itself which are more >> efficient than marshalling/unmarshalling. >> >> Thoughts? >> >>> >>> Sanne >>> >>> >>> Any other ideas? >>> Cheers, > > On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > >> Hi all, >> >> As mentioned in >> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, >> in paralell to the switch to Equivalent* collections, I was also working >> on being able to pass metadata into Infinispan caches. This is done to >> better support the ability to store custom metadata in Infinispan >> without the need of extra wrappers. So, the idea is that >> InternalCacheEntry instances will have a a reference to this Metadata. >> >> One of that metadata is version, which I've been using as test bed to >> see if clients could pass succesfully version information via metadata. >> As you already know, Hot Rod requires to store version information. >> Before, this was stored in a class called CacheValue alongside the value >> itself, but the work I've done in [1]
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 1:11 PM, Dan Berindei wrote: > > > > On Mon, Apr 8, 2013 at 1:44 PM, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > > > > On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > > > >> All sounds very good. One important thing to consider is that the > >> reference to Metadata passed in by the client app will be tied to the ICE > >> for the entire lifespan of the ICE. You'll need to think about a > >> defensive copy or some other form of making the Metadata immutable (by the > >> user application, at least) the moment it is passed in. > > > > ^ Excellent point, it could be a nightmare if users could change the > > metadata reference by the ICE at will. I'll have a think on how to best > > achieve this. > > ^ The metadata is gonna have to be marshalled somehow to ship to other nodes, > so that could be a way to achieve it, by enforcing this somehow. When the > cache receives it, it can marshaller/unmarshall it to make a copy > > > If Metadata is just an interface, nothing is stopping the user from > implementing maxIdle() to return Random.maxLong(). Besides, local caches need > to support Metadata as well, and we shouldn't force > serialization/deserialization for local caches. > > So I think we'd be better off documenting that Metadata objects should not > change after they are inserted in the cache, just like keys and values. > > > One way would be to make Metadata extend Serializable, but not keen on that. > Another would be to somehow force the interface to define the Externalizer to > use (i.e. an interface method like getExternalizer()), but that's akward when > it comes to unmarshalling… what about forcing the Metadata object to be > provided with a @SerializeWith annotation? > > Any other ideas? > > > Why force anything? I think Metadata instances should be treated just like > keys and values, so they should be able to use Externalizers (via > @SerializeWith), Serializable, or Externalizable, depending on the user's > requirements. ^ I agree. What do you think of my suggestion in the other email to separate both concerns and somehow enforce a copy of the object to be provided instead? > > > > > > Cheers, > > > >> > >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > >> > >>> Hi all, > >>> > >>> As mentioned in > >>> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, > >>> in paralell to the switch to Equivalent* collections, I was also working > >>> on being able to pass metadata into Infinispan caches. This is done to > >>> better support the ability to store custom metadata in Infinispan without > >>> the need of extra wrappers. So, the idea is that InternalCacheEntry > >>> instances will have a a reference to this Metadata. > >>> > >>> One of that metadata is version, which I've been using as test bed to see > >>> if clients could pass succesfully version information via metadata. As > >>> you already know, Hot Rod requires to store version information. Before, > >>> this was stored in a class called CacheValue alongside the value itself, > >>> but the work I've done in [1], this is passed via the new API I've added > >>> in [2]. > >>> > >>> So, I'd like to get some thoughts on this new API. I hope that with these > >>> new put/replace versions, we can get rid of the nightmare which is all > >>> the other put/replace calls taking lifespan and/or maxIdle information. > >>> In the end, I think there should be two basic puts: > >>> > >>> - put(K, V) > >>> - put(K, V, Metadata) > >>> > >>> And their equivalents. > >>> > >>> IMPORTANT NOTE 1: The implementation details are bound to change, because > >>> the entire Metadata needs to be stored in InternalCacheEntry, not just > >>> version, lifespan..etc. I'll further develop the implementation once I > >>> get into adding more metadata, i.e. when working on interoperability with > >>> REST. So, don't pay too much attention to the implementation itself, > >>> focus on the AdvancedCache API itself and let's refine that. > >>> > >>> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, so > >>> please let's avoid discussing it in this email thread. Once I have a more > >>> final version I'll send an email about it. > >>> > >>> Apart from working on enhancements to the API, I'm now carry on tackling > >>> the interoperability work with aim to have an initial version of the > >>> Embedded <-> Hot Rod interoperability as first step. Once that's in, it > >>> can be released to get early feedback while the rest of interoperability > >>> modes are developed. > >>> > >>> Cheers, > >>> > >>> [1] > >>> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d > >>> [2] > >>> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d#L10R313 > >>> -- > >>> Galder Zamarreño > >>> gal...@redhat.com > >>> twitter.com/galderz > >>> > >>> Project Lead
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On 8 April 2013 12:06, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:56 PM, Sanne Grinovero wrote: > >> >> >> >> On 8 April 2013 11:44, Galder Zamarreño wrote: >> >> On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: >> >> > >> > On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: >> > >> >> All sounds very good. One important thing to consider is that the >> >> reference to Metadata passed in by the client app will be tied to the ICE >> >> for the entire lifespan of the ICE. You'll need to think about a >> >> defensive copy or some other form of making the Metadata immutable (by >> >> the user application, at least) the moment it is passed in. >> > >> > ^ Excellent point, it could be a nightmare if users could change the >> > metadata reference by the ICE at will. I'll have a think on how to best >> > achieve this. >> >> ^ The metadata is gonna have to be marshalled somehow to ship to other >> nodes, so that could be a way to achieve it, by enforcing this somehow. When >> the cache receives it, it can marshaller/unmarshall it to make a copy >> >> One way would be to make Metadata extend Serializable, but not keen on that. >> Another would be to somehow force the interface to define the Externalizer >> to use (i.e. an interface method like getExternalizer()), but that's akward >> when it comes to unmarshalling… what about forcing the Metadata object to be >> provided with a @SerializeWith annotation? >> >> Why is getExternalizer() awkward for unmarshalling? > > ^ Because you don't have an instance yet, so what's the Externalizer for it? > IOW, there's no much point to doing that, simply register it depending on > your desire: > https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers That's what I would expect. > >> I would expect you to have the marshaller already known during >> deserialization. > > You would, as long as you follow the instructions in > https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers > >> Agreed that extensing Serializable is not a good idea. >> >> Are you thinking about the impact on CacheStore(s) and state transfer? > > ^ What about it in particular? > >> Eviction of no longer used metadata ? > > ^ Since the metadata is part of the entry, it'd initially go when the entry > is evicted. We might wanna leave it around in some cases… but it'd be for > other use cases. I thought the plan was to have entries refer to the metadata, but that different entries sharing the same metadata would point to the same instance. So this metadata needs to be stored separately in the CacheStore, preloaded as appropriate, transferred during state transfer, passivated when convenient and cleaned up when no longer referred to. Am I wrong? Seems you plan to store a copy of the metadata within each ICE. > > I'm also considering separating the serialization/marshalling concerns from > the defensive copying concerns. IOW, add a copy() method to the Metadata > interface, or have a separate interface for those externally provided objects > that require to be defensive copied. IOW, do something like what Scala Case > Classes do with their copy() method, but without the issues of clone… I need > to investigate this further to come up with a nice solution. > > One positive side to splitting both concerns is speed. A Metadata > implementation might have ways to make a copy of itself which are more > efficient than marshalling/unmarshalling. > > Thoughts? > >> >> Sanne >> >> >> Any other ideas? >> >> > >> > Cheers, >> > >> >> >> >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: >> >> >> >>> Hi all, >> >>> >> >>> As mentioned in >> >>> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, >> >>> in paralell to the switch to Equivalent* collections, I was also working >> >>> on being able to pass metadata into Infinispan caches. This is done to >> >>> better support the ability to store custom metadata in Infinispan >> >>> without the need of extra wrappers. So, the idea is that >> >>> InternalCacheEntry instances will have a a reference to this Metadata. >> >>> >> >>> One of that metadata is version, which I've been using as test bed to >> >>> see if clients could pass succesfully version information via metadata. >> >>> As you already know, Hot Rod requires to store version information. >> >>> Before, this was stored in a class called CacheValue alongside the value >> >>> itself, but the work I've done in [1], this is passed via the new API >> >>> I've added in [2]. >> >>> >> >>> So, I'd like to get some thoughts on this new API. I hope that with >> >>> these new put/replace versions, we can get rid of the nightmare which is >> >>> all the other put/replace calls taking lifespan and/or maxIdle >> >>> information. In the end, I think there should be two basic puts: >> >>> >> >>> - put(K, V) >> >>> - put(K, V, Metadata) >> >>> >> >>> And their equivalents. >> >>>
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Mon, Apr 8, 2013 at 1:44 PM, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > > > > On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > > > >> All sounds very good. One important thing to consider is that the > reference to Metadata passed in by the client app will be tied to the ICE > for the entire lifespan of the ICE. You'll need to think about a defensive > copy or some other form of making the Metadata immutable (by the user > application, at least) the moment it is passed in. > > > > ^ Excellent point, it could be a nightmare if users could change the > metadata reference by the ICE at will. I'll have a think on how to best > achieve this. > > ^ The metadata is gonna have to be marshalled somehow to ship to other > nodes, so that could be a way to achieve it, by enforcing this somehow. > When the cache receives it, it can marshaller/unmarshall it to make a copy > > If Metadata is just an interface, nothing is stopping the user from implementing maxIdle() to return Random.maxLong(). Besides, local caches need to support Metadata as well, and we shouldn't force serialization/deserialization for local caches. So I think we'd be better off documenting that Metadata objects should not change after they are inserted in the cache, just like keys and values. > One way would be to make Metadata extend Serializable, but not keen on > that. Another would be to somehow force the interface to define the > Externalizer to use (i.e. an interface method like getExternalizer()), but > that's akward when it comes to unmarshalling… what about forcing the > Metadata object to be provided with a @SerializeWith annotation? > > Any other ideas? > > Why force anything? I think Metadata instances should be treated just like keys and values, so they should be able to use Externalizers (via @SerializeWith), Serializable, or Externalizable, depending on the user's requirements. > > > > Cheers, > > > >> > >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > >> > >>> Hi all, > >>> > >>> As mentioned in > http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, > in paralell to the switch to Equivalent* collections, I was also working on > being able to pass metadata into Infinispan caches. This is done to better > support the ability to store custom metadata in Infinispan without the need > of extra wrappers. So, the idea is that InternalCacheEntry instances will > have a a reference to this Metadata. > >>> > >>> One of that metadata is version, which I've been using as test bed to > see if clients could pass succesfully version information via metadata. As > you already know, Hot Rod requires to store version information. Before, > this was stored in a class called CacheValue alongside the value itself, > but the work I've done in [1], this is passed via the new API I've added in > [2]. > >>> > >>> So, I'd like to get some thoughts on this new API. I hope that with > these new put/replace versions, we can get rid of the nightmare which is > all the other put/replace calls taking lifespan and/or maxIdle information. > In the end, I think there should be two basic puts: > >>> > >>> - put(K, V) > >>> - put(K, V, Metadata) > >>> > >>> And their equivalents. > >>> > >>> IMPORTANT NOTE 1: The implementation details are bound to change, > because the entire Metadata needs to be stored in InternalCacheEntry, not > just version, lifespan..etc. I'll further develop the implementation once I > get into adding more metadata, i.e. when working on interoperability with > REST. So, don't pay too much attention to the implementation itself, focus > on the AdvancedCache API itself and let's refine that. > >>> > >>> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, > so please let's avoid discussing it in this email thread. Once I have a > more final version I'll send an email about it. > >>> > >>> Apart from working on enhancements to the API, I'm now carry on > tackling the interoperability work with aim to have an initial version of > the Embedded <-> Hot Rod interoperability as first step. Once that's in, it > can be released to get early feedback while the rest of interoperability > modes are developed. > >>> > >>> Cheers, > >>> > >>> [1] > https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d > >>> [2] > https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d#L10R313 > >>> -- > >>> Galder Zamarreño > >>> gal...@redhat.com > >>> twitter.com/galderz > >>> > >>> Project Lead, Escalante > >>> http://escalante.io > >>> > >>> Engineer, Infinispan > >>> http://infinispan.org > >>> > >>> > >>> ___ > >>> infinispan-dev mailing list > >>> infinispan-dev@lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> -- > >> Manik Surtani > >> ma...@jboss.org > >> twitter.com/maniksurtani > >> > >> Platform Architect, JBoss Data Grid > >>
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 12:56 PM, Sanne Grinovero wrote: > > > > On 8 April 2013 11:44, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > > > > On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > > > >> All sounds very good. One important thing to consider is that the > >> reference to Metadata passed in by the client app will be tied to the ICE > >> for the entire lifespan of the ICE. You'll need to think about a > >> defensive copy or some other form of making the Metadata immutable (by the > >> user application, at least) the moment it is passed in. > > > > ^ Excellent point, it could be a nightmare if users could change the > > metadata reference by the ICE at will. I'll have a think on how to best > > achieve this. > > ^ The metadata is gonna have to be marshalled somehow to ship to other nodes, > so that could be a way to achieve it, by enforcing this somehow. When the > cache receives it, it can marshaller/unmarshall it to make a copy > > One way would be to make Metadata extend Serializable, but not keen on that. > Another would be to somehow force the interface to define the Externalizer to > use (i.e. an interface method like getExternalizer()), but that's akward when > it comes to unmarshalling… what about forcing the Metadata object to be > provided with a @SerializeWith annotation? > > Why is getExternalizer() awkward for unmarshalling? ^ Because you don't have an instance yet, so what's the Externalizer for it? IOW, there's no much point to doing that, simply register it depending on your desire: https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers > I would expect you to have the marshaller already known during > deserialization. You would, as long as you follow the instructions in https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+Defined+Externalizers > Agreed that extensing Serializable is not a good idea. > > Are you thinking about the impact on CacheStore(s) and state transfer? ^ What about it in particular? > Eviction of no longer used metadata ? ^ Since the metadata is part of the entry, it'd initially go when the entry is evicted. We might wanna leave it around in some cases… but it'd be for other use cases. I'm also considering separating the serialization/marshalling concerns from the defensive copying concerns. IOW, add a copy() method to the Metadata interface, or have a separate interface for those externally provided objects that require to be defensive copied. IOW, do something like what Scala Case Classes do with their copy() method, but without the issues of clone… I need to investigate this further to come up with a nice solution. One positive side to splitting both concerns is speed. A Metadata implementation might have ways to make a copy of itself which are more efficient than marshalling/unmarshalling. Thoughts? > > Sanne > > > Any other ideas? > > > > > Cheers, > > > >> > >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > >> > >>> Hi all, > >>> > >>> As mentioned in > >>> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, > >>> in paralell to the switch to Equivalent* collections, I was also working > >>> on being able to pass metadata into Infinispan caches. This is done to > >>> better support the ability to store custom metadata in Infinispan without > >>> the need of extra wrappers. So, the idea is that InternalCacheEntry > >>> instances will have a a reference to this Metadata. > >>> > >>> One of that metadata is version, which I've been using as test bed to see > >>> if clients could pass succesfully version information via metadata. As > >>> you already know, Hot Rod requires to store version information. Before, > >>> this was stored in a class called CacheValue alongside the value itself, > >>> but the work I've done in [1], this is passed via the new API I've added > >>> in [2]. > >>> > >>> So, I'd like to get some thoughts on this new API. I hope that with these > >>> new put/replace versions, we can get rid of the nightmare which is all > >>> the other put/replace calls taking lifespan and/or maxIdle information. > >>> In the end, I think there should be two basic puts: > >>> > >>> - put(K, V) > >>> - put(K, V, Metadata) > >>> > >>> And their equivalents. > >>> > >>> IMPORTANT NOTE 1: The implementation details are bound to change, because > >>> the entire Metadata needs to be stored in InternalCacheEntry, not just > >>> version, lifespan..etc. I'll further develop the implementation once I > >>> get into adding more metadata, i.e. when working on interoperability with > >>> REST. So, don't pay too much attention to the implementation itself, > >>> focus on the AdvancedCache API itself and let's refine that. > >>> > >>> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, so > >>> please let's avoid discussing it in this email thread. Once I have a
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On 8 April 2013 11:44, Galder Zamarreño wrote: > > On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > > > > On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > > > >> All sounds very good. One important thing to consider is that the > reference to Metadata passed in by the client app will be tied to the ICE > for the entire lifespan of the ICE. You'll need to think about a defensive > copy or some other form of making the Metadata immutable (by the user > application, at least) the moment it is passed in. > > > > ^ Excellent point, it could be a nightmare if users could change the > metadata reference by the ICE at will. I'll have a think on how to best > achieve this. > > ^ The metadata is gonna have to be marshalled somehow to ship to other > nodes, so that could be a way to achieve it, by enforcing this somehow. > When the cache receives it, it can marshaller/unmarshall it to make a copy > > One way would be to make Metadata extend Serializable, but not keen on > that. Another would be to somehow force the interface to define the > Externalizer to use (i.e. an interface method like getExternalizer()), but > that's akward when it comes to unmarshalling… what about forcing the > Metadata object to be provided with a @SerializeWith annotation? > Why is getExternalizer() awkward for unmarshalling? I would expect you to have the marshaller already known during deserialization. Agreed that extensing Serializable is not a good idea. Are you thinking about the impact on CacheStore(s) and state transfer? Eviction of no longer used metadata ? Sanne > > Any other ideas? > > > > > Cheers, > > > >> > >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > >> > >>> Hi all, > >>> > >>> As mentioned in > http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, > in paralell to the switch to Equivalent* collections, I was also working on > being able to pass metadata into Infinispan caches. This is done to better > support the ability to store custom metadata in Infinispan without the need > of extra wrappers. So, the idea is that InternalCacheEntry instances will > have a a reference to this Metadata. > >>> > >>> One of that metadata is version, which I've been using as test bed to > see if clients could pass succesfully version information via metadata. As > you already know, Hot Rod requires to store version information. Before, > this was stored in a class called CacheValue alongside the value itself, > but the work I've done in [1], this is passed via the new API I've added in > [2]. > >>> > >>> So, I'd like to get some thoughts on this new API. I hope that with > these new put/replace versions, we can get rid of the nightmare which is > all the other put/replace calls taking lifespan and/or maxIdle information. > In the end, I think there should be two basic puts: > >>> > >>> - put(K, V) > >>> - put(K, V, Metadata) > >>> > >>> And their equivalents. > >>> > >>> IMPORTANT NOTE 1: The implementation details are bound to change, > because the entire Metadata needs to be stored in InternalCacheEntry, not > just version, lifespan..etc. I'll further develop the implementation once I > get into adding more metadata, i.e. when working on interoperability with > REST. So, don't pay too much attention to the implementation itself, focus > on the AdvancedCache API itself and let's refine that. > >>> > >>> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, > so please let's avoid discussing it in this email thread. Once I have a > more final version I'll send an email about it. > >>> > >>> Apart from working on enhancements to the API, I'm now carry on > tackling the interoperability work with aim to have an initial version of > the Embedded <-> Hot Rod interoperability as first step. Once that's in, it > can be released to get early feedback while the rest of interoperability > modes are developed. > >>> > >>> Cheers, > >>> > >>> [1] > https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d > >>> [2] > https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d#L10R313 > >>> -- > >>> Galder Zamarreño > >>> gal...@redhat.com > >>> twitter.com/galderz > >>> > >>> Project Lead, Escalante > >>> http://escalante.io > >>> > >>> Engineer, Infinispan > >>> http://infinispan.org > >>> > >>> > >>> ___ > >>> infinispan-dev mailing list > >>> infinispan-dev@lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> -- > >> Manik Surtani > >> ma...@jboss.org > >> twitter.com/maniksurtani > >> > >> Platform Architect, JBoss Data Grid > >> http://red.ht/data-grid > >> > >> > >> ___ > >> infinispan-dev mailing list > >> infinispan-dev@lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > -- > > Galder Zamarreño > > gal...@redhat.com > > twitter.com/galderz > > > > Project Lead, Escalante >
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 12:35 PM, Galder Zamarreño wrote: > > On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > >> All sounds very good. One important thing to consider is that the reference >> to Metadata passed in by the client app will be tied to the ICE for the >> entire lifespan of the ICE. You'll need to think about a defensive copy or >> some other form of making the Metadata immutable (by the user application, >> at least) the moment it is passed in. > > ^ Excellent point, it could be a nightmare if users could change the metadata > reference by the ICE at will. I'll have a think on how to best achieve this. ^ The metadata is gonna have to be marshalled somehow to ship to other nodes, so that could be a way to achieve it, by enforcing this somehow. When the cache receives it, it can marshaller/unmarshall it to make a copy One way would be to make Metadata extend Serializable, but not keen on that. Another would be to somehow force the interface to define the Externalizer to use (i.e. an interface method like getExternalizer()), but that's akward when it comes to unmarshalling… what about forcing the Metadata object to be provided with a @SerializeWith annotation? Any other ideas? > > Cheers, > >> >> On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: >> >>> Hi all, >>> >>> As mentioned in >>> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, in >>> paralell to the switch to Equivalent* collections, I was also working on >>> being able to pass metadata into Infinispan caches. This is done to better >>> support the ability to store custom metadata in Infinispan without the need >>> of extra wrappers. So, the idea is that InternalCacheEntry instances will >>> have a a reference to this Metadata. >>> >>> One of that metadata is version, which I've been using as test bed to see >>> if clients could pass succesfully version information via metadata. As you >>> already know, Hot Rod requires to store version information. Before, this >>> was stored in a class called CacheValue alongside the value itself, but the >>> work I've done in [1], this is passed via the new API I've added in [2]. >>> >>> So, I'd like to get some thoughts on this new API. I hope that with these >>> new put/replace versions, we can get rid of the nightmare which is all the >>> other put/replace calls taking lifespan and/or maxIdle information. In the >>> end, I think there should be two basic puts: >>> >>> - put(K, V) >>> - put(K, V, Metadata) >>> >>> And their equivalents. >>> >>> IMPORTANT NOTE 1: The implementation details are bound to change, because >>> the entire Metadata needs to be stored in InternalCacheEntry, not just >>> version, lifespan..etc. I'll further develop the implementation once I get >>> into adding more metadata, i.e. when working on interoperability with REST. >>> So, don't pay too much attention to the implementation itself, focus on the >>> AdvancedCache API itself and let's refine that. >>> >>> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, so >>> please let's avoid discussing it in this email thread. Once I have a more >>> final version I'll send an email about it. >>> >>> Apart from working on enhancements to the API, I'm now carry on tackling >>> the interoperability work with aim to have an initial version of the >>> Embedded <-> Hot Rod interoperability as first step. Once that's in, it can >>> be released to get early feedback while the rest of interoperability modes >>> are developed. >>> >>> Cheers, >>> >>> [1] >>> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d >>> [2] >>> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d#L10R313 >>> -- >>> Galder Zamarreño >>> gal...@redhat.com >>> twitter.com/galderz >>> >>> Project Lead, Escalante >>> http://escalante.io >>> >>> Engineer, Infinispan >>> http://infinispan.org >>> >>> >>> ___ >>> infinispan-dev mailing list >>> infinispan-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Manik Surtani >> ma...@jboss.org >> twitter.com/maniksurtani >> >> Platform Architect, JBoss Data Grid >> http://red.ht/data-grid >> >> >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Galder Zamarreño > gal...@redhat.com > twitter.com/galderz > > Project Lead, Escalante > http://escalante.io > > Engineer, Infinispan > http://infinispan.org > -- Galder Zamarreño gal...@redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
On Apr 8, 2013, at 11:17 AM, Manik Surtani wrote: > All sounds very good. One important thing to consider is that the reference > to Metadata passed in by the client app will be tied to the ICE for the > entire lifespan of the ICE. You'll need to think about a defensive copy or > some other form of making the Metadata immutable (by the user application, at > least) the moment it is passed in. ^ Excellent point, it could be a nightmare if users could change the metadata reference by the ICE at will. I'll have a think on how to best achieve this. Cheers, > > On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > >> Hi all, >> >> As mentioned in >> http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, in >> paralell to the switch to Equivalent* collections, I was also working on >> being able to pass metadata into Infinispan caches. This is done to better >> support the ability to store custom metadata in Infinispan without the need >> of extra wrappers. So, the idea is that InternalCacheEntry instances will >> have a a reference to this Metadata. >> >> One of that metadata is version, which I've been using as test bed to see if >> clients could pass succesfully version information via metadata. As you >> already know, Hot Rod requires to store version information. Before, this >> was stored in a class called CacheValue alongside the value itself, but the >> work I've done in [1], this is passed via the new API I've added in [2]. >> >> So, I'd like to get some thoughts on this new API. I hope that with these >> new put/replace versions, we can get rid of the nightmare which is all the >> other put/replace calls taking lifespan and/or maxIdle information. In the >> end, I think there should be two basic puts: >> >> - put(K, V) >> - put(K, V, Metadata) >> >> And their equivalents. >> >> IMPORTANT NOTE 1: The implementation details are bound to change, because >> the entire Metadata needs to be stored in InternalCacheEntry, not just >> version, lifespan..etc. I'll further develop the implementation once I get >> into adding more metadata, i.e. when working on interoperability with REST. >> So, don't pay too much attention to the implementation itself, focus on the >> AdvancedCache API itself and let's refine that. >> >> IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, so >> please let's avoid discussing it in this email thread. Once I have a more >> final version I'll send an email about it. >> >> Apart from working on enhancements to the API, I'm now carry on tackling the >> interoperability work with aim to have an initial version of the Embedded >> <-> Hot Rod interoperability as first step. Once that's in, it can be >> released to get early feedback while the rest of interoperability modes are >> developed. >> >> Cheers, >> >> [1] >> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d >> [2] >> https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d#L10R313 >> -- >> Galder Zamarreño >> gal...@redhat.com >> twitter.com/galderz >> >> Project Lead, Escalante >> http://escalante.io >> >> Engineer, Infinispan >> http://infinispan.org >> >> >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > ma...@jboss.org > twitter.com/maniksurtani > > Platform Architect, JBoss Data Grid > http://red.ht/data-grid > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño gal...@redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] AdvancedCache.put with Metadata parameter
All sounds very good. One important thing to consider is that the reference to Metadata passed in by the client app will be tied to the ICE for the entire lifespan of the ICE. You'll need to think about a defensive copy or some other form of making the Metadata immutable (by the user application, at least) the moment it is passed in. On 8 Apr 2013, at 09:24, Galder Zamarreño wrote: > Hi all, > > As mentioned in > http://lists.jboss.org/pipermail/infinispan-dev/2013-March/012348.html, in > paralell to the switch to Equivalent* collections, I was also working on > being able to pass metadata into Infinispan caches. This is done to better > support the ability to store custom metadata in Infinispan without the need > of extra wrappers. So, the idea is that InternalCacheEntry instances will > have a a reference to this Metadata. > > One of that metadata is version, which I've been using as test bed to see if > clients could pass succesfully version information via metadata. As you > already know, Hot Rod requires to store version information. Before, this was > stored in a class called CacheValue alongside the value itself, but the work > I've done in [1], this is passed via the new API I've added in [2]. > > So, I'd like to get some thoughts on this new API. I hope that with these new > put/replace versions, we can get rid of the nightmare which is all the other > put/replace calls taking lifespan and/or maxIdle information. In the end, I > think there should be two basic puts: > > - put(K, V) > - put(K, V, Metadata) > > And their equivalents. > > IMPORTANT NOTE 1: The implementation details are bound to change, because the > entire Metadata needs to be stored in InternalCacheEntry, not just version, > lifespan..etc. I'll further develop the implementation once I get into adding > more metadata, i.e. when working on interoperability with REST. So, don't pay > too much attention to the implementation itself, focus on the AdvancedCache > API itself and let's refine that. > > IMPORTANT NOTE 2: The interoperability work in commit in [1] is WIP, so > please let's avoid discussing it in this email thread. Once I have a more > final version I'll send an email about it. > > Apart from working on enhancements to the API, I'm now carry on tackling the > interoperability work with aim to have an initial version of the Embedded <-> > Hot Rod interoperability as first step. Once that's in, it can be released to > get early feedback while the rest of interoperability modes are developed. > > Cheers, > > [1] > https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d > [2] > https://github.com/galderz/infinispan/commit/a35956fe291d2b2dc3b7fa7bf44d8965ffb1a54d#L10R313 > -- > Galder Zamarreño > gal...@redhat.com > twitter.com/galderz > > Project Lead, Escalante > http://escalante.io > > Engineer, Infinispan > http://infinispan.org > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
