Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-18 Thread Manik Surtani
On 15 Apr 2011, at 17:43, Olaf Bergner wrote: >>> >>> Plus, I fear rehashing would have to be aware of wheter it is dealing with >>> relocating a large object chunk or a "regular" value. > See above: if a node leaves the cluster rehashing might relocate large > object's A 's chunk to a node th

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-15 Thread Olaf Bergner
Am 15.04.11 16:54, schrieb Manik Surtani: > On 4 Apr 2011, at 11:01, Olaf Bergner wrote: > >>> What we somehow need to avoid is chunks ending up in nodes that do not >>> have enough memory to store them, and that could complicate things. >> Definitely. What about replication, for instance? Does INF

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-15 Thread Manik Surtani
On 4 Apr 2011, at 11:01, Olaf Bergner wrote: >> >> What we somehow need to avoid is chunks ending up in nodes that do not >> have enough memory to store them, and that could complicate things. > > Definitely. What about replication, for instance? Does INFINISPAN use the > replication mechanis

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-15 Thread Manik Surtani
+1, I don't think we should over-complicate by mandating that chunks are on different nodes. Let the distribution code handle this. If, at a later date, we see that the system frequently fails due to too many chunks on certain nodes, we can revisit. But that would be an implementation detail.

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-05 Thread Olaf Bergner
Am 05.04.11 10:27, schrieb Galder Zamarreño: > On Apr 4, 2011, at 11:50 AM, Olaf Bergner wrote: > >> Hi Galder, >> >> Original-Nachricht >>> Datum: Mon, 4 Apr 2011 11:01:21 +0200 >>> Von: "Galder Zamarreño" >>> Actually, it's not that simple, it needs to be adaptive but it might b

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-05 Thread Galder Zamarreño
On Apr 4, 2011, at 11:50 AM, Olaf Bergner wrote: > Hi Galder, > > Original-Nachricht >> Datum: Mon, 4 Apr 2011 11:01:21 +0200 >> Von: "Galder Zamarreño" >> Actually, it's not that simple, it needs to be adaptive but it might be >> going into the territory of virtual nodes and

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Sanne Grinovero
> I haven't thought deeply about the implications for concurrent writes, > though. Is it possible to lock keys outside of a transactional context? If > so, this might be a solution for reading and writing. No, it's not possible to lock anything out of transaction; the most obvious reason is th

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Dan Berindei
> >> > 3. The design suggests to use a fresh UUID as the key for each new >> > chunk. While this in all likelihood gives us a unique new key for each >> > chunk I currently fail to see how that guarantees that this key maps to >> > a node that is different from all the nodes already used to store c

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Olaf Bergner
Hi Galder, Original-Nachricht > Datum: Mon, 4 Apr 2011 11:29:06 +0200 > Von: "Galder Zamarreño" > > in size. Anyway, it should be easy to refactor if reusing > > PutKeyValueCommand should prove viable. > > The only reason it reads the previous value is to return it as part of

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Olaf Bergner
Hi Galder, Original-Nachricht > Datum: Mon, 4 Apr 2011 11:01:21 +0200 > Von: "Galder Zamarreño" > Actually, it's not that simple, it needs to be adaptive but it might be > going into the territory of virtual nodes and sizing of virtual nodes. The > key thing of choosing the node

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Olaf Bergner
Hi Galder, thanks for your input. See my comments below: Original-Nachricht > Datum: Mon, 4 Apr 2011 10:09:39 +0200 > Von: "Galder Zamarreño" > > H, the initial step in writeToKey() is to create an map entry for the > metadata, so the internal writeToKey() could indeed cr

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Sanne Grinovero
I don't think you should make it too complex by looking at available memory, you have the same issue when storing many different keys in Infinispan in any mode, but we never worry about this, relying instead on the spreading quality of the hash function, and of course the available total heap size

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Galder Zamarreño
On Mar 31, 2011, at 7:46 AM, Olaf Bergner wrote: > Am 30.03.11 02:32, schrieb Elias Ross: >> I think it'd be BEST if you could support both models. I would add: >> >> interface Cache { >> /** >>* Returns a new or existing LargeObject object for the following key. >>* @throws ClassCastE

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Galder Zamarreño
On Apr 4, 2011, at 10:09 AM, Galder Zamarreño wrote: > Hi Olaf, > > See below for comments: > > On Mar 29, 2011, at 12:49 PM, Olaf Bergner wrote: > >> >> >> 3. The design suggests to use a fresh UUID as the key for each new >> chunk. While this in all likelihood gives us a unique new key fo

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-04-04 Thread Galder Zamarreño
Hi Olaf, See below for comments: On Mar 29, 2011, at 12:49 PM, Olaf Bergner wrote: > I've started working on ISPN-78 - Large Object Support - closely > following Manik's design document > http://community.jboss.org/wiki/LargeObjectSupport. As a starting point > I'm currently trying to impleme

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-03-30 Thread Olaf Bergner
Am 30.03.11 02:32, schrieb Elias Ross: > I think it'd be BEST if you could support both models. I would add: > > interface Cache { >/** > * Returns a new or existing LargeObject object for the following key. > * @throws ClassCastException if the key exists and is not a LargeObject. >

Re: [infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-03-29 Thread Elias Ross
On Tue, Mar 29, 2011 at 3:49 AM, Olaf Bergner wrote: > I've started working on ISPN-78 - Large Object Support - closely > following Manik's design document > http://community.jboss.org/wiki/LargeObjectSupport. As a starting point > I'm currently trying to implement > > OutputStream writeToKey(K k

[infinispan-dev] [ISPN-78] Alternative interface for writing large objects

2011-03-29 Thread Olaf Bergner
I've started working on ISPN-78 - Large Object Support - closely following Manik's design document http://community.jboss.org/wiki/LargeObjectSupport. As a starting point I'm currently trying to implement OutputStream writeToKey(K key), which, for the time being, I chose to declare on Advanced