Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

On Jan 27, 2014, at 2:02 PM, Pedro Ruivo  wrote:

> On 01/27/2014 01:38 PM, Dan Berindei wrote:
>> 
>> 
>> 
>> On Mon, Jan 27, 2014 at 2:43 PM, Pedro Ruivo > > wrote:
>> 
>> 
>> 
>>On 01/27/2014 12:26 PM, Emmanuel Bernard wrote:
>>> I'd be curious to see performance tests on Pedro's approach (ie walk
>>> through the entire data key set to find the matching elements of
>>a given
>>> group). That might be fast enough but that looks quite scary
>>compared to
>>> a single lookup.
>> 
>>I would prefer to have a performance hit than a map of sets (group name
>>=> set of keys). I also think that keep this map synchronized with the
>>keys in data container will not be easy...
>> 
>> 
>> Sure, I would prefer the simpler implementation as well. But if changing
>> an application to use groups instead of atomic maps will change the
>> processing time of a request from 1ms to 1s, I'm pretty sure users will
>> prefer to keep use the atomic maps :)
> 
> you don't need to change the application. we can implement the 
> AtomicHashMap interface on top of grouping :D
> 
> I'm expecting a negative performance impact but not that high. Also, 
> with the current implementation, FGAHM performs a copy for writing... 
> anyway, we should test and see how it goes :)

+1. We can keep both around for a while and only drop FGAM iff grouping does 
the job right.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

On Jan 27, 2014, at 1:38 PM, Dan Berindei  wrote:

> 
> 
> 
> On Mon, Jan 27, 2014 at 2:43 PM, Pedro Ruivo  wrote:
> 
> 
> On 01/27/2014 12:26 PM, Emmanuel Bernard wrote:
> > I'd be curious to see performance tests on Pedro's approach (ie walk
> > through the entire data key set to find the matching elements of a given
> > group). That might be fast enough but that looks quite scary compared to
> > a single lookup.
> 
> I would prefer to have a performance hit than a map of sets (group name
> => set of keys). I also think that keep this map synchronized with the
> keys in data container will not be easy...
> 
> Sure, I would prefer the simpler implementation as well. But if changing an 
> application to use groups instead of atomic maps will change the processing 
> time of a request from 1ms to 1s, I'm pretty sure users will prefer to keep 
> use the atomic maps :)

+1
Also the Map> Emmanuel mentions is something that already 
exists within the (FG)AM.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Pedro Ruivo]

On 01/27/2014 01:38 PM, Dan Berindei wrote:
>
>
>
> On Mon, Jan 27, 2014 at 2:43 PM, Pedro Ruivo  > wrote:
>
>
>
> On 01/27/2014 12:26 PM, Emmanuel Bernard wrote:
>  > I'd be curious to see performance tests on Pedro's approach (ie walk
>  > through the entire data key set to find the matching elements of
> a given
>  > group). That might be fast enough but that looks quite scary
> compared to
>  > a single lookup.
>
> I would prefer to have a performance hit than a map of sets (group name
> => set of keys). I also think that keep this map synchronized with the
> keys in data container will not be easy...
>
>
> Sure, I would prefer the simpler implementation as well. But if changing
> an application to use groups instead of atomic maps will change the
> processing time of a request from 1ms to 1s, I'm pretty sure users will
> prefer to keep use the atomic maps :)

you don't need to change the application. we can implement the 
AtomicHashMap interface on top of grouping :D

I'm expecting a negative performance impact but not that high. Also, 
with the current implementation, FGAHM performs a copy for writing... 
anyway, we should test and see how it goes :)

>
>
>  >
>  > Any doc explaining how FGAM is broken in transactions for curiosity.
>  >
>
> well, the map is not isolated, so you can see the updates from other
> transactions immediately (https://issues.jboss.org/browse/ISPN-3932)
>
>
> Do you know if AtomicMap is affected, too?

I haven't tested yet, but I'm assuming the worst (i.e. yes, it is 
affected). I'm trying to find a way to fix it without destroying 
anything else :(

>
> It also does not work when you enable write skew check with optimistic
> transactions (we have a JIRA somewhere)
>
>
> I assume you mean https://issues.jboss.org/browse/ISPN-3939 ?
> This looks like it also affects AtomicMap, so the only workaround is to
> use pessimistic locking.

that is cross-site replication...

I mean to this: https://issues.jboss.org/browse/ISPN-2729
that is originated because we don't support version in Deltas

>
>
>
>  > On Mon 2014-01-27 11:54, Pedro Ruivo wrote:
>  >>
>  >>
>  >> On 01/27/2014 09:52 AM, Sanne Grinovero wrote:
>  >>> On 27 January 2014 09:38, Pedro Ruivo  > wrote:
>  
>  
>   On 01/27/2014 09:20 AM, Sanne Grinovero wrote:
>  > On 23 January 2014 18:03, Dan Berindei
> mailto:dan.berin...@gmail.com>> wrote:
>  >>
>  >> On 22 Jan 2014 16:10, "Pedro Ruivo"  > wrote:
>  >>>
>  >>>
>  >>>
>  >>> On 01/22/2014 01:58 PM, Dan Berindei wrote:
>  
>  
>   It would also require us to keep a Set for each group,
> with the keys
>   associated with that group. As such, I'm not sure it would
> be a lot
>   easier to implement (correctly) than FineGrainedAtomicMap.
>  
>  
>  >>>
>  >>> Dan, I didn't understand why do we need to keep a Set.
> Can you
>  >>> elaborate?
>  >>
>  >>
>  >> We'd need some way to keep track of the keys that are part
> of the group,
>  >> iterating over the entire cache for every getGroup() call
> would be way too
>  >> slow.
>  >
>  > Right, and load all entries from any CacheStore too :-/
>  
>   IMO, I prefer to iterate over the data container and cache
> loader when
>   it is needed than keep the Set for each group. I think the
> memory
>   will thank you
>  >>>
>  >>> Of course. I'm just highlighting how importand Dan's comment is,
>  >>> because we obviously don' t want to load everything from
> CacheStore.
>  >>> So, tracking which entries are part of the group is essential:
>  >>> failing this, I'm still skeptical about why the Grouping API is a
>  >>> better replacement than FGAM.
>  >>
>  >> I have one reason: FGAM does not work inside transactions...
>  >>
>  >>>
>  >>> Sanne
>  >>> ___
>  >>> infinispan-dev mailing list
>  >>> infinispan-dev@lists.jboss.org
> 
>  >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>  >>>
>  >> ___
>  >> infinispan-dev mailing list
>  >> infinispan-dev@lists.jboss.org
> 
>  >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>  > ___
>  > infinispan-dev mailing list
>  > infinispan-dev@lists.jboss.org
> 
>  > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>  >
>  

Re: [infinispan-dev] Integration between HotRod and OGM

[Dan Berindei]

On Mon, Jan 27, 2014 at 2:43 PM, Pedro Ruivo  wrote:

>
>
> On 01/27/2014 12:26 PM, Emmanuel Bernard wrote:
> > I'd be curious to see performance tests on Pedro's approach (ie walk
> > through the entire data key set to find the matching elements of a given
> > group). That might be fast enough but that looks quite scary compared to
> > a single lookup.
>
> I would prefer to have a performance hit than a map of sets (group name
> => set of keys). I also think that keep this map synchronized with the
> keys in data container will not be easy...
>

Sure, I would prefer the simpler implementation as well. But if changing an
application to use groups instead of atomic maps will change the processing
time of a request from 1ms to 1s, I'm pretty sure users will prefer to keep
use the atomic maps :)


> >
> > Any doc explaining how FGAM is broken in transactions for curiosity.
> >
>
> well, the map is not isolated, so you can see the updates from other
> transactions immediately (https://issues.jboss.org/browse/ISPN-3932)
>
>
Do you know if AtomicMap is affected, too?



> It also does not work when you enable write skew check with optimistic
> transactions (we have a JIRA somewhere)
>

I assume you mean https://issues.jboss.org/browse/ISPN-3939 ?
This looks like it also affects AtomicMap, so the only workaround is to use
pessimistic locking.



>
> > On Mon 2014-01-27 11:54, Pedro Ruivo wrote:
> >>
> >>
> >> On 01/27/2014 09:52 AM, Sanne Grinovero wrote:
> >>> On 27 January 2014 09:38, Pedro Ruivo  wrote:
> 
> 
>  On 01/27/2014 09:20 AM, Sanne Grinovero wrote:
> > On 23 January 2014 18:03, Dan Berindei 
> wrote:
> >>
> >> On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:
> >>>
> >>>
> >>>
> >>> On 01/22/2014 01:58 PM, Dan Berindei wrote:
> 
> 
>  It would also require us to keep a Set for each group, with
> the keys
>  associated with that group. As such, I'm not sure it would be a
> lot
>  easier to implement (correctly) than FineGrainedAtomicMap.
> 
> 
> >>>
> >>> Dan, I didn't understand why do we need to keep a Set. Can you
> >>> elaborate?
> >>
> >>
> >> We'd need some way to keep track of the keys that are part of the
> group,
> >> iterating over the entire cache for every getGroup() call would be
> way too
> >> slow.
> >
> > Right, and load all entries from any CacheStore too :-/
> 
>  IMO, I prefer to iterate over the data container and cache loader when
>  it is needed than keep the Set for each group. I think the memory
>  will thank you
> >>>
> >>> Of course. I'm just highlighting how importand Dan's comment is,
> >>> because we obviously don' t want to load everything from CacheStore.
> >>> So, tracking which entries are part of the group is essential:
> >>> failing this, I'm still skeptical about why the Grouping API is a
> >>> better replacement than FGAM.
> >>
> >> I have one reason: FGAM does not work inside transactions...
> >>
> >>>
> >>> Sanne
> >>> ___
> >>> infinispan-dev mailing list
> >>> infinispan-dev@lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >> ___
> >> infinispan-dev mailing list
> >> infinispan-dev@lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > ___
> > infinispan-dev mailing list
> > infinispan-dev@lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Pedro Ruivo]

On 01/27/2014 12:26 PM, Emmanuel Bernard wrote:
> I'd be curious to see performance tests on Pedro's approach (ie walk
> through the entire data key set to find the matching elements of a given
> group). That might be fast enough but that looks quite scary compared to
> a single lookup.

I would prefer to have a performance hit than a map of sets (group name 
=> set of keys). I also think that keep this map synchronized with the 
keys in data container will not be easy...

>
> Any doc explaining how FGAM is broken in transactions for curiosity.
>

well, the map is not isolated, so you can see the updates from other 
transactions immediately (https://issues.jboss.org/browse/ISPN-3932)

It also does not work when you enable write skew check with optimistic 
transactions (we have a JIRA somewhere)

> On Mon 2014-01-27 11:54, Pedro Ruivo wrote:
>>
>>
>> On 01/27/2014 09:52 AM, Sanne Grinovero wrote:
>>> On 27 January 2014 09:38, Pedro Ruivo  wrote:


 On 01/27/2014 09:20 AM, Sanne Grinovero wrote:
> On 23 January 2014 18:03, Dan Berindei  wrote:
>>
>> On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:
>>>
>>>
>>>
>>> On 01/22/2014 01:58 PM, Dan Berindei wrote:


 It would also require us to keep a Set for each group, with the keys
 associated with that group. As such, I'm not sure it would be a lot
 easier to implement (correctly) than FineGrainedAtomicMap.


>>>
>>> Dan, I didn't understand why do we need to keep a Set. Can you
>>> elaborate?
>>
>>
>> We'd need some way to keep track of the keys that are part of the group,
>> iterating over the entire cache for every getGroup() call would be way 
>> too
>> slow.
>
> Right, and load all entries from any CacheStore too :-/

 IMO, I prefer to iterate over the data container and cache loader when
 it is needed than keep the Set for each group. I think the memory
 will thank you
>>>
>>> Of course. I'm just highlighting how importand Dan's comment is,
>>> because we obviously don' t want to load everything from CacheStore.
>>> So, tracking which entries are part of the group is essential:
>>> failing this, I'm still skeptical about why the Grouping API is a
>>> better replacement than FGAM.
>>
>> I have one reason: FGAM does not work inside transactions...
>>
>>>
>>> Sanne
>>> ___
>>> infinispan-dev mailing list
>>> infinispan-dev@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

I'd be curious to see performance tests on Pedro's approach (ie walk
through the entire data key set to find the matching elements of a given
group). That might be fast enough but that looks quite scary compared to
a single lookup.

Any doc explaining how FGAM is broken in transactions for curiosity.

On Mon 2014-01-27 11:54, Pedro Ruivo wrote:
> 
> 
> On 01/27/2014 09:52 AM, Sanne Grinovero wrote:
> > On 27 January 2014 09:38, Pedro Ruivo  wrote:
> >>
> >>
> >> On 01/27/2014 09:20 AM, Sanne Grinovero wrote:
> >>> On 23 January 2014 18:03, Dan Berindei  wrote:
> 
>  On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:
> >
> >
> >
> > On 01/22/2014 01:58 PM, Dan Berindei wrote:
> >>
> >>
> >> It would also require us to keep a Set for each group, with the keys
> >> associated with that group. As such, I'm not sure it would be a lot
> >> easier to implement (correctly) than FineGrainedAtomicMap.
> >>
> >>
> >
> > Dan, I didn't understand why do we need to keep a Set. Can you
> > elaborate?
> 
> 
>  We'd need some way to keep track of the keys that are part of the group,
>  iterating over the entire cache for every getGroup() call would be way 
>  too
>  slow.
> >>>
> >>> Right, and load all entries from any CacheStore too :-/
> >>
> >> IMO, I prefer to iterate over the data container and cache loader when
> >> it is needed than keep the Set for each group. I think the memory
> >> will thank you
> >
> > Of course. I'm just highlighting how importand Dan's comment is,
> > because we obviously don' t want to load everything from CacheStore.
> > So, tracking which entries are part of the group is essential:
> > failing this, I'm still skeptical about why the Grouping API is a
> > better replacement than FGAM.
> 
> I have one reason: FGAM does not work inside transactions...
> 
> >
> > Sanne
> > ___
> > infinispan-dev mailing list
> > infinispan-dev@lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Pedro Ruivo]

On 01/27/2014 09:52 AM, Sanne Grinovero wrote:
> On 27 January 2014 09:38, Pedro Ruivo  wrote:
>>
>>
>> On 01/27/2014 09:20 AM, Sanne Grinovero wrote:
>>> On 23 January 2014 18:03, Dan Berindei  wrote:

 On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:
>
>
>
> On 01/22/2014 01:58 PM, Dan Berindei wrote:
>>
>>
>> It would also require us to keep a Set for each group, with the keys
>> associated with that group. As such, I'm not sure it would be a lot
>> easier to implement (correctly) than FineGrainedAtomicMap.
>>
>>
>
> Dan, I didn't understand why do we need to keep a Set. Can you
> elaborate?


 We'd need some way to keep track of the keys that are part of the group,
 iterating over the entire cache for every getGroup() call would be way too
 slow.
>>>
>>> Right, and load all entries from any CacheStore too :-/
>>
>> IMO, I prefer to iterate over the data container and cache loader when
>> it is needed than keep the Set for each group. I think the memory
>> will thank you
>
> Of course. I'm just highlighting how importand Dan's comment is,
> because we obviously don' t want to load everything from CacheStore.
> So, tracking which entries are part of the group is essential:
> failing this, I'm still skeptical about why the Grouping API is a
> better replacement than FGAM.

I have one reason: FGAM does not work inside transactions...

>
> Sanne
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Sanne Grinovero]

On 27 January 2014 09:38, Pedro Ruivo  wrote:
>
>
> On 01/27/2014 09:20 AM, Sanne Grinovero wrote:
>> On 23 January 2014 18:03, Dan Berindei  wrote:
>>>
>>> On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:



 On 01/22/2014 01:58 PM, Dan Berindei wrote:
>
>
> It would also require us to keep a Set for each group, with the keys
> associated with that group. As such, I'm not sure it would be a lot
> easier to implement (correctly) than FineGrainedAtomicMap.
>
>

 Dan, I didn't understand why do we need to keep a Set. Can you
 elaborate?
>>>
>>>
>>> We'd need some way to keep track of the keys that are part of the group,
>>> iterating over the entire cache for every getGroup() call would be way too
>>> slow.
>>
>> Right, and load all entries from any CacheStore too :-/
>
> IMO, I prefer to iterate over the data container and cache loader when
> it is needed than keep the Set for each group. I think the memory
> will thank you

Of course. I'm just highlighting how importand Dan's comment is,
because we obviously don' t want to load everything from CacheStore.
So, tracking which entries are part of the group is essential:
failing this, I'm still skeptical about why the Grouping API is a
better replacement than FGAM.

Sanne
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Pedro Ruivo]

On 01/27/2014 09:20 AM, Sanne Grinovero wrote:
> On 23 January 2014 18:03, Dan Berindei  wrote:
>>
>> On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:
>>>
>>>
>>>
>>> On 01/22/2014 01:58 PM, Dan Berindei wrote:


 It would also require us to keep a Set for each group, with the keys
 associated with that group. As such, I'm not sure it would be a lot
 easier to implement (correctly) than FineGrainedAtomicMap.


>>>
>>> Dan, I didn't understand why do we need to keep a Set. Can you
>>> elaborate?
>>
>>
>> We'd need some way to keep track of the keys that are part of the group,
>> iterating over the entire cache for every getGroup() call would be way too
>> slow.
>
> Right, and load all entries from any CacheStore too :-/

IMO, I prefer to iterate over the data container and cache loader when 
it is needed than keep the Set for each group. I think the memory 
will thank you

> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Sanne Grinovero]

On 23 January 2014 18:03, Dan Berindei  wrote:
>
> On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:
>>
>>
>>
>> On 01/22/2014 01:58 PM, Dan Berindei wrote:
>> >
>> >
>> > It would also require us to keep a Set for each group, with the keys
>> > associated with that group. As such, I'm not sure it would be a lot
>> > easier to implement (correctly) than FineGrainedAtomicMap.
>> >
>> >
>>
>> Dan, I didn't understand why do we need to keep a Set. Can you
>> elaborate?
>
>
> We'd need some way to keep track of the keys that are part of the group,
> iterating over the entire cache for every getGroup() call would be way too
> slow.

Right, and load all entries from any CacheStore too :-/
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Dan Berindei]

On 22 Jan 2014 16:10, "Pedro Ruivo"  wrote:
>
>
>
> On 01/22/2014 01:58 PM, Dan Berindei wrote:
> >
> >
> > It would also require us to keep a Set for each group, with the keys
> > associated with that group. As such, I'm not sure it would be a lot
> > easier to implement (correctly) than FineGrainedAtomicMap.
> >
> >
>
> Dan, I didn't understand why do we need to keep a Set. Can you
> elaborate?


We'd need some way to keep track of the keys that are part of the group,
iterating over the entire cache for every getGroup() call would be way too
slow.
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

On 22 Jan 2014, at 15:11, Emmanuel Bernard  wrote:

>> So you don't use the AtomicMap(vs FGAM) at all? Is there any place in which 
>> you require a lock in the whole map to be acquired?
> 
> I will be not right now.

Hum, it should read: It will be. But not right now.___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Pedro Ruivo]

On 01/22/2014 01:58 PM, Dan Berindei wrote:
>
>
> It would also require us to keep a Set for each group, with the keys
> associated with that group. As such, I'm not sure it would be a lot
> easier to implement (correctly) than FineGrainedAtomicMap.
>
>

Dan, I didn't understand why do we need to keep a Set. Can you 
elaborate?
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

On 22 Jan 2014, at 14:48, Mircea Markus  wrote:

> 
> On Jan 22, 2014, at 1:26 PM, Emmanuel Bernard  wrote:
> 
>> Conceptually I like the grouping API better than AtomicMap as I don’t have 
>> to rely on a specific Infinispan type.
>> 
>> We do use FineGrainedAtomicMap both for the entity and the association 
>> persistence (not AtomicMap).
> 
> So you don't use the AtomicMap(vs FGAM) at all? Is there any place in which 
> you require a lock in the whole map to be acquired?

I will be not right now.
It seems that Sanne moved both entity and association to use the FGAM to lower 
the lock contentions. I don’t quite remember if that was fully intentional or a 
side effect. Intuitively, I’d see us use AM for entities but that’s not the 
case.

> 
>> It is particularly critical for how we store the association navigation 
>> information. I don’t want one update to literally prevent the whole 
>> association from being updated. This is the same semantic a RDBMS has and 
>> that’s why Manik and I designed the FGAM requirements.
>> 
>> So my question is what are the differences between the grouping API and the 
>> FGAM in particular for:
>> 
>> - the amount of data sent back and forth (seems like grouping is sending the 
>> data naturally per key as “delta compared to the group"
> 
> - we'll only send the (group, key, value) for every group write. The same 
> amount of info is sent for an FGAM.put
> 
>> - the locking level when a new entry is added to the FGAM / Grouping API
>> - the locking level when a new entry is removed to the FGAM / Grouping API
>> - the locking level when a new entry is updated to the FGAM / Grouping API
> 
> - in the case of grouping the lock object is the tuple (group, key), so the 
> lock granularity is the same as FGAM, which under the hood build an synthetic 
> lock object based on (FGAM, innerKey)

Don’t FGAM uses the “group” level lock when it create / delete the group? What 
about create / delta keys in the group (which have to be added to a list of 
keys in the group)?

> 
>> - the overall network verbosity
> 
> - same
> 
>> - does grouping offer the same repeatable read protection that AtomicMap 
>> offers within a transaction?
> 
> - yes, and it actually has a clearer semantics, as the grouping API is 
> entirely built on top of the basic cache operations, instead of being a first 
> class citizen with it's own transaction semantics.
> 
>> 
>> I think retrying as a transaction workaround is quite fragile. We can offer 
>> it as a solution but supporting or encouraging it is another story. Unless 
>> each OGM nodes do behave like a transaction but that would be wrong. I am 
>> also concerned about reading data form a group that are inconsistent.
> 
> The grouping API I'm suggesting offers an nicer alternative to the (FG)AM 
> approach that would also work over HotRod. ATM there's no TX support for 
> HotRod so it seems like to be able to support the HotRod and OGM integration 
> fully, we'd need HR transactions as well. Do you think the integration can be 
> done in steps: 
> 1. add grouping over hotrod and integrate with OGM
> 2. add tx and make the integration use it
> 
> Or we should wait till 2. and then proceed with the integration?

We can / should to it in two steps as long as we mark it as toy / non data safe 
in the documentation until step 2 is done.

___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

Sent from my iPhone

> On 22 Jan 2014, at 13:58, Dan Berindei  wrote:
> 
> 
> 
> 
>> On Tue, Jan 21, 2014 at 4:07 PM, Mircea Markus  wrote:
>> Hi Emmanuel,
>> 
>> Just had a good chat with Davide on this and one solution to overcome the 
>> shortcoming you mentioned in the above email would be to enhance the hotrod 
>> client to support grouping:
>> 
>> RemoteClient.put(G g, K k, V v); //first param is the group
>> RemoteClinet.getGroup(G g) : Map;
> 
> I think you'd also need RemoteClient.get(G g, K k), as in embedded mode the 
> group is included in the key.

Yes

> 
>  
>> 
>> It requires an enhancement on our local grouping API: 
>> EmbeddedCache.getGroup(G). This is something useful for us in a broader 
>> context, as it is the step needed to be able to deprecated AtomicMaps and 
>> get suggest them being replaced with Grouping.
> 
> It would also require us to keep a Set for each group, with the keys 
> associated with that group. As such, I'm not sure it would be a lot easier to 
> implement (correctly) than FineGrainedAtomicMap.
> 
>> 
>> This approach still has some limitations compared to the current embedded 
>> integration:
>> - performance caused by the lack of transactions: this means increased TCP 
>> chattiness between the Hot Rod client and the server.
>> - you'd have to handle atomicity, potentially by retrying an operation
>> 
>> What do you think?
>> 
>> 
>> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
>> 
>> >
>> > On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  
>> > wrote:
>> >
>> >> It's an interesting approach that would work fine-ish for entities
>> >> assuming the Hot Rod client is multi threaded and assuming the client
>> >> uses Future to parallelize the calls.
>> >
>> > The Java Hotrod client is both multithreaded and exposes an Async API.
>> >
>> >>
>> >> But it won't work for associations as we have them designed today.
>> >> Each association - or more precisely the query results to go from an
>> >> entity A1 to the list of entities B associated to it - is represented by
>> >> an AtomicMap.
>> >> Each entry in this map does correspond to an entry in the association.
>> >>
>> >> While we can "guess" the column names and build from the metadata the
>> >> list of composed keys for entities, we cannot do the same for
>> >> associations as the key is literally the (composite) id of the
>> >> association and we cannot guess that most of the time (we can in very
>> >> pathological cases).
>> >> We could imagine that we list the association row keys in a special
>> >> entry to work around that but this approach is just as problematic and
>> >> is conceptually the same.
>> >> The only solution would be to lock the whole association for each
>> >> operation and I guess impose some versioning / optimistic lock.
>> >>
>> >> That is not a pattern that scales sufficiently from my experience.
>> >
>> > I think so too :-)
>> >
>> >> That's the problem with interconnected data :)
>> >>
>> >> Emmanuel
>> >>
>> >> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>> >>> Neither the grouping API nor the AtomicMap work over hotrod.
>> >>> Between the grouping API and AtomicMap, I think the one that would make 
>> >>> more sense migrating is the grouping API.
>> >>> One way or the other, I think the hotrod protocol would require an 
>> >>> enhancement - mind raising a JIRA for that?
>> >>> For now I guess you can sacrifice performance and always sending the 
>> >>> entire object across on every update instead of only the deltas?
>> >>>
>> >>> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
>> >>> wrote:
>> >>>
>>  Someone mentioned the grouping API as some sort of alternative to
>>  AtomicMap. Maybe we should use that?
>>  Note that if we don't have a fine-grained approach we will need to
>>  make sure we *copy* the complex data structure upon reads to mimic
>>  proper transaction isolation.
>> 
>>  On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>> > On 12 November 2013 14:54, Emmanuel Bernard  
>> > wrote:
>> >> On the transaction side, we can start without them.
>> >
>> > +1 on omitting transactions for now.
>> >
>> > And on the missing AtomicMaps, I hope the Infinispan will want to 
>> > implement it?
>> > Would be good to eventually converge on similar featuresets on remote
>> > vs embedded APIs.
>> >
>> > I know the embedded version relies on batching/transactions, but I
>> > guess we could obtain a similar effect with some ad-hoc commands in
>> > Hot Rod?
>> >
>> > Sanne
>> >
>> >>
>> >> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>> >>> Hi,
>> >>> I'm working on the integration between HotRod and OGM.
>> >>>
>> >>> We already have a dialect for Inifinispan and I'm trying to follow 
>> >>> the same
>> >>> logic.
>> >>> At the moment I'm having two problems:
>> >>>
>> >>> 1) In the Infinispan dialect we are using the AtomicMap and the
>> >>> At

Re: [infinispan-dev] Integration between HotRod and OGM

[Dan Berindei]

On Tue, Jan 21, 2014 at 4:07 PM, Mircea Markus  wrote:

> Hi Emmanuel,
>
> Just had a good chat with Davide on this and one solution to overcome the
> shortcoming you mentioned in the above email would be to enhance the hotrod
> client to support grouping:
>
> RemoteClient.put(G g, K k, V v); //first param is the group
> RemoteClinet.getGroup(G g) : Map;
>

I think you'd also need RemoteClient.get(G g, K k), as in embedded mode the
group is included in the key.



>
> It requires an enhancement on our local grouping API:
> EmbeddedCache.getGroup(G). This is something useful for us in a broader
> context, as it is the step needed to be able to deprecated AtomicMaps and
> get suggest them being replaced with Grouping.
>

It would also require us to keep a Set for each group, with the keys
associated with that group. As such, I'm not sure it would be a lot easier
to implement (correctly) than FineGrainedAtomicMap.


> This approach still has some limitations compared to the current embedded
> integration:
> - performance caused by the lack of transactions: this means increased TCP
> chattiness between the Hot Rod client and the server.
> - you'd have to handle atomicity, potentially by retrying an operation
>
> What do you think?
>
>
> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
>
> >
> > On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard 
> wrote:
> >
> >> It's an interesting approach that would work fine-ish for entities
> >> assuming the Hot Rod client is multi threaded and assuming the client
> >> uses Future to parallelize the calls.
> >
> > The Java Hotrod client is both multithreaded and exposes an Async API.
> >
> >>
> >> But it won't work for associations as we have them designed today.
> >> Each association - or more precisely the query results to go from an
> >> entity A1 to the list of entities B associated to it - is represented by
> >> an AtomicMap.
> >> Each entry in this map does correspond to an entry in the association.
> >>
> >> While we can "guess" the column names and build from the metadata the
> >> list of composed keys for entities, we cannot do the same for
> >> associations as the key is literally the (composite) id of the
> >> association and we cannot guess that most of the time (we can in very
> >> pathological cases).
> >> We could imagine that we list the association row keys in a special
> >> entry to work around that but this approach is just as problematic and
> >> is conceptually the same.
> >> The only solution would be to lock the whole association for each
> >> operation and I guess impose some versioning / optimistic lock.
> >>
> >> That is not a pattern that scales sufficiently from my experience.
> >
> > I think so too :-)
> >
> >> That's the problem with interconnected data :)
> >>
> >> Emmanuel
> >>
> >> On Mon 2013-11-18 23:05, Mircea Markus wrote:
> >>> Neither the grouping API nor the AtomicMap work over hotrod.
> >>> Between the grouping API and AtomicMap, I think the one that would
> make more sense migrating is the grouping API.
> >>> One way or the other, I think the hotrod protocol would require an
> enhancement - mind raising a JIRA for that?
> >>> For now I guess you can sacrifice performance and always sending the
> entire object across on every update instead of only the deltas?
> >>>
> >>> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard 
> wrote:
> >>>
>  Someone mentioned the grouping API as some sort of alternative to
>  AtomicMap. Maybe we should use that?
>  Note that if we don't have a fine-grained approach we will need to
>  make sure we *copy* the complex data structure upon reads to mimic
>  proper transaction isolation.
> 
>  On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
> > On 12 November 2013 14:54, Emmanuel Bernard 
> wrote:
> >> On the transaction side, we can start without them.
> >
> > +1 on omitting transactions for now.
> >
> > And on the missing AtomicMaps, I hope the Infinispan will want to
> implement it?
> > Would be good to eventually converge on similar featuresets on remote
> > vs embedded APIs.
> >
> > I know the embedded version relies on batching/transactions, but I
> > guess we could obtain a similar effect with some ad-hoc commands in
> > Hot Rod?
> >
> > Sanne
> >
> >>
> >> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
> >>> Hi,
> >>> I'm working on the integration between HotRod and OGM.
> >>>
> >>> We already have a dialect for Inifinispan and I'm trying to follow
> the same
> >>> logic.
> >>> At the moment I'm having two problems:
> >>>
> >>> 1) In the Infinispan dialect we are using the AtomicMap and the
> >>> AtomicMapLookup but this classes don't work with the RemoteCache.
> Is there
> >>> an equivalent for HotRod?
> >>>
> >>> 2) As far as I know HotRod does not support transactions. I've
> found a link
> >>> to a branch on Mircea repository:
> >>> https://github.com/

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

On Jan 22, 2014, at 1:26 PM, Emmanuel Bernard  wrote:

> Conceptually I like the grouping API better than AtomicMap as I don’t have to 
> rely on a specific Infinispan type.
> 
> We do use FineGrainedAtomicMap both for the entity and the association 
> persistence (not AtomicMap).

So you don't use the AtomicMap(vs FGAM) at all? Is there any place in which you 
require a lock in the whole map to be acquired?

> It is particularly critical for how we store the association navigation 
> information. I don’t want one update to literally prevent the whole 
> association from being updated. This is the same semantic a RDBMS has and 
> that’s why Manik and I designed the FGAM requirements.
> 
> So my question is what are the differences between the grouping API and the 
> FGAM in particular for:
> 
> - the amount of data sent back and forth (seems like grouping is sending the 
> data naturally per key as “delta compared to the group"

- we'll only send the (group, key, value) for every group write. The same 
amount of info is sent for an FGAM.put

> - the locking level when a new entry is added to the FGAM / Grouping API
> - the locking level when a new entry is removed to the FGAM / Grouping API
> - the locking level when a new entry is updated to the FGAM / Grouping API

- in the case of grouping the lock object is the tuple (group, key), so the 
lock granularity is the same as FGAM, which under the hood build an synthetic 
lock object based on (FGAM, innerKey)

> - the overall network verbosity

- same

> - does grouping offer the same repeatable read protection that AtomicMap 
> offers within a transaction?

- yes, and it actually has a clearer semantics, as the grouping API is entirely 
built on top of the basic cache operations, instead of being a first class 
citizen with it's own transaction semantics.

> 
> I think retrying as a transaction workaround is quite fragile. We can offer 
> it as a solution but supporting or encouraging it is another story. Unless 
> each OGM nodes do behave like a transaction but that would be wrong. I am 
> also concerned about reading data form a group that are inconsistent.

The grouping API I'm suggesting offers an nicer alternative to the (FG)AM 
approach that would also work over HotRod. ATM there's no TX support for HotRod 
so it seems like to be able to support the HotRod and OGM integration fully, 
we'd need HR transactions as well. Do you think the integration can be done in 
steps: 
1. add grouping over hotrod and integrate with OGM
2. add tx and make the integration use it

Or we should wait till 2. and then proceed with the integration?

> 
> Emmanuel
> 
> On 21 Jan 2014, at 16:07, Mircea Markus  wrote:
> 
>> Hi Emmanuel,
>> 
>> Just had a good chat with Davide on this and one solution to overcome the 
>> shortcoming you mentioned in the above email would be to enhance the hotrod 
>> client to support grouping:
>> 
>> RemoteClient.put(G g, K k, V v); //first param is the group
>> RemoteClinet.getGroup(G g) : Map;
>> 
>> It requires an enhancement on our local grouping API: 
>> EmbeddedCache.getGroup(G). This is something useful for us in a broader 
>> context, as it is the step needed to be able to deprecated AtomicMaps and 
>> get suggest them being replaced with Grouping. 
>> 
>> This approach still has some limitations compared to the current embedded 
>> integration: 
>> - performance caused by the lack of transactions: this means increased TCP 
>> chattiness between the Hot Rod client and the server. 
>> - you'd have to handle atomicity, potentially by retrying an operation
>> 
>> What do you think?
>> 
>> 
>> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
>> 
>>> 
>>> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  
>>> wrote:
>>> 
 It's an interesting approach that would work fine-ish for entities
 assuming the Hot Rod client is multi threaded and assuming the client
 uses Future to parallelize the calls.
>>> 
>>> The Java Hotrod client is both multithreaded and exposes an Async API.
>>> 
 
 But it won't work for associations as we have them designed today.
 Each association - or more precisely the query results to go from an
 entity A1 to the list of entities B associated to it - is represented by
 an AtomicMap.
 Each entry in this map does correspond to an entry in the association.
 
 While we can "guess" the column names and build from the metadata the
 list of composed keys for entities, we cannot do the same for
 associations as the key is literally the (composite) id of the
 association and we cannot guess that most of the time (we can in very
 pathological cases).
 We could imagine that we list the association row keys in a special
 entry to work around that but this approach is just as problematic and
 is conceptually the same.
 The only solution would be to lock the whole association for each
 operation and I guess impose some versioning / optimistic lock.
 
>

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

On the other hand if the group feature adds a way to:
- get all the keys,
- get a subset of the keys based on a filter

Infinispan will be able to start supporting use cases restricted to Cassandra 
in the past (esp around time series).

I am assuming groups offer offer a way to add / change and remove a key from / 
to a group without having to load all the group or even the group keys.

On 22 Jan 2014, at 14:33, Emmanuel Bernard  wrote:

> BTW query support on groups (by entry of each group) is an interesting non 
> covered use case today.
> 
> On 22 Jan 2014, at 14:26, Emmanuel Bernard  wrote:
> 
>> Conceptually I like the grouping API better than AtomicMap as I don’t have 
>> to rely on a specific Infinispan type.
>> 
>> We do use FineGrainedAtomicMap both for the entity and the association 
>> persistence (not AtomicMap). It is particularly critical for how we store 
>> the association navigation information. I don’t want one update to literally 
>> prevent the whole association from being updated. This is the same semantic 
>> a RDBMS has and that’s why Manik and I designed the FGAM requirements.
>> 
>> So my question is what are the differences between the grouping API and the 
>> FGAM in particular for:
>> 
>> - the amount of data sent back and forth (seems like grouping is sending the 
>> data naturally per key as “delta compared to the group"
>> - the locking level when a new entry is added to the FGAM / Grouping API
>> - the locking level when a new entry is removed to the FGAM / Grouping API
>> - the locking level when a new entry is updated to the FGAM / Grouping API
>> - the overall network verbosity
>> - does grouping offer the same repeatable read protection that AtomicMap 
>> offers within a transaction?
>> 
>> I think retrying as a transaction workaround is quite fragile. We can offer 
>> it as a solution but supporting or encouraging it is another story. Unless 
>> each OGM nodes do behave like a transaction but that would be wrong. I am 
>> also concerned about reading data form a group that are inconsistent.
>> 
>> Emmanuel
>> 
>> On 21 Jan 2014, at 16:07, Mircea Markus  wrote:
>> 
>>> Hi Emmanuel,
>>> 
>>> Just had a good chat with Davide on this and one solution to overcome the 
>>> shortcoming you mentioned in the above email would be to enhance the hotrod 
>>> client to support grouping:
>>> 
>>> RemoteClient.put(G g, K k, V v); //first param is the group
>>> RemoteClinet.getGroup(G g) : Map;
>>> 
>>> It requires an enhancement on our local grouping API: 
>>> EmbeddedCache.getGroup(G). This is something useful for us in a broader 
>>> context, as it is the step needed to be able to deprecated AtomicMaps and 
>>> get suggest them being replaced with Grouping. 
>>> 
>>> This approach still has some limitations compared to the current embedded 
>>> integration: 
>>> - performance caused by the lack of transactions: this means increased TCP 
>>> chattiness between the Hot Rod client and the server. 
>>> - you'd have to handle atomicity, potentially by retrying an operation
>>> 
>>> What do you think?
>>> 
>>> 
>>> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
>>> 
 
 On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  
 wrote:
 
> It's an interesting approach that would work fine-ish for entities
> assuming the Hot Rod client is multi threaded and assuming the client
> uses Future to parallelize the calls.
 
 The Java Hotrod client is both multithreaded and exposes an Async API.
 
> 
> But it won't work for associations as we have them designed today.
> Each association - or more precisely the query results to go from an
> entity A1 to the list of entities B associated to it - is represented by
> an AtomicMap.
> Each entry in this map does correspond to an entry in the association.
> 
> While we can "guess" the column names and build from the metadata the
> list of composed keys for entities, we cannot do the same for
> associations as the key is literally the (composite) id of the
> association and we cannot guess that most of the time (we can in very
> pathological cases).
> We could imagine that we list the association row keys in a special
> entry to work around that but this approach is just as problematic and
> is conceptually the same.
> The only solution would be to lock the whole association for each
> operation and I guess impose some versioning / optimistic lock.
> 
> That is not a pattern that scales sufficiently from my experience.
 
 I think so too :-)
 
> That's the problem with interconnected data :)
> 
> Emmanuel
> 
> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>> Neither the grouping API nor the AtomicMap work over hotrod.
>> Between the grouping API and AtomicMap, I think the one that would make 
>> more sense migrating is the grouping API.
>> One way or the other, I think the hotrod protocol would requir

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

BTW query support on groups (by entry of each group) is an interesting non 
covered use case today.

On 22 Jan 2014, at 14:26, Emmanuel Bernard  wrote:

> Conceptually I like the grouping API better than AtomicMap as I don’t have to 
> rely on a specific Infinispan type.
> 
> We do use FineGrainedAtomicMap both for the entity and the association 
> persistence (not AtomicMap). It is particularly critical for how we store the 
> association navigation information. I don’t want one update to literally 
> prevent the whole association from being updated. This is the same semantic a 
> RDBMS has and that’s why Manik and I designed the FGAM requirements.
> 
> So my question is what are the differences between the grouping API and the 
> FGAM in particular for:
> 
> - the amount of data sent back and forth (seems like grouping is sending the 
> data naturally per key as “delta compared to the group"
> - the locking level when a new entry is added to the FGAM / Grouping API
> - the locking level when a new entry is removed to the FGAM / Grouping API
> - the locking level when a new entry is updated to the FGAM / Grouping API
> - the overall network verbosity
> - does grouping offer the same repeatable read protection that AtomicMap 
> offers within a transaction?
> 
> I think retrying as a transaction workaround is quite fragile. We can offer 
> it as a solution but supporting or encouraging it is another story. Unless 
> each OGM nodes do behave like a transaction but that would be wrong. I am 
> also concerned about reading data form a group that are inconsistent.
> 
> Emmanuel
> 
> On 21 Jan 2014, at 16:07, Mircea Markus  wrote:
> 
>> Hi Emmanuel,
>> 
>> Just had a good chat with Davide on this and one solution to overcome the 
>> shortcoming you mentioned in the above email would be to enhance the hotrod 
>> client to support grouping:
>> 
>> RemoteClient.put(G g, K k, V v); //first param is the group
>> RemoteClinet.getGroup(G g) : Map;
>> 
>> It requires an enhancement on our local grouping API: 
>> EmbeddedCache.getGroup(G). This is something useful for us in a broader 
>> context, as it is the step needed to be able to deprecated AtomicMaps and 
>> get suggest them being replaced with Grouping. 
>> 
>> This approach still has some limitations compared to the current embedded 
>> integration: 
>> - performance caused by the lack of transactions: this means increased TCP 
>> chattiness between the Hot Rod client and the server. 
>> - you'd have to handle atomicity, potentially by retrying an operation
>> 
>> What do you think?
>> 
>> 
>> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
>> 
>>> 
>>> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  
>>> wrote:
>>> 
 It's an interesting approach that would work fine-ish for entities
 assuming the Hot Rod client is multi threaded and assuming the client
 uses Future to parallelize the calls.
>>> 
>>> The Java Hotrod client is both multithreaded and exposes an Async API.
>>> 
 
 But it won't work for associations as we have them designed today.
 Each association - or more precisely the query results to go from an
 entity A1 to the list of entities B associated to it - is represented by
 an AtomicMap.
 Each entry in this map does correspond to an entry in the association.
 
 While we can "guess" the column names and build from the metadata the
 list of composed keys for entities, we cannot do the same for
 associations as the key is literally the (composite) id of the
 association and we cannot guess that most of the time (we can in very
 pathological cases).
 We could imagine that we list the association row keys in a special
 entry to work around that but this approach is just as problematic and
 is conceptually the same.
 The only solution would be to lock the whole association for each
 operation and I guess impose some versioning / optimistic lock.
 
 That is not a pattern that scales sufficiently from my experience.
>>> 
>>> I think so too :-)
>>> 
 That's the problem with interconnected data :)
 
 Emmanuel
 
 On Mon 2013-11-18 23:05, Mircea Markus wrote:
> Neither the grouping API nor the AtomicMap work over hotrod.
> Between the grouping API and AtomicMap, I think the one that would make 
> more sense migrating is the grouping API.
> One way or the other, I think the hotrod protocol would require an 
> enhancement - mind raising a JIRA for that?
> For now I guess you can sacrifice performance and always sending the 
> entire object across on every update instead of only the deltas?
> 
> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
> wrote:
> 
>> Someone mentioned the grouping API as some sort of alternative to
>> AtomicMap. Maybe we should use that?
>> Note that if we don't have a fine-grained approach we will need to
>> make sure we *copy* the complex data structure upon reads to mimic

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

Conceptually I like the grouping API better than AtomicMap as I don’t have to 
rely on a specific Infinispan type.

We do use FineGrainedAtomicMap both for the entity and the association 
persistence (not AtomicMap). It is particularly critical for how we store the 
association navigation information. I don’t want one update to literally 
prevent the whole association from being updated. This is the same semantic a 
RDBMS has and that’s why Manik and I designed the FGAM requirements.

So my question is what are the differences between the grouping API and the 
FGAM in particular for:

- the amount of data sent back and forth (seems like grouping is sending the 
data naturally per key as “delta compared to the group"
- the locking level when a new entry is added to the FGAM / Grouping API
- the locking level when a new entry is removed to the FGAM / Grouping API
- the locking level when a new entry is updated to the FGAM / Grouping API
- the overall network verbosity
- does grouping offer the same repeatable read protection that AtomicMap offers 
within a transaction?

I think retrying as a transaction workaround is quite fragile. We can offer it 
as a solution but supporting or encouraging it is another story. Unless each 
OGM nodes do behave like a transaction but that would be wrong. I am also 
concerned about reading data form a group that are inconsistent.

Emmanuel

On 21 Jan 2014, at 16:07, Mircea Markus  wrote:

> Hi Emmanuel,
> 
> Just had a good chat with Davide on this and one solution to overcome the 
> shortcoming you mentioned in the above email would be to enhance the hotrod 
> client to support grouping:
> 
> RemoteClient.put(G g, K k, V v); //first param is the group
> RemoteClinet.getGroup(G g) : Map;
> 
> It requires an enhancement on our local grouping API: 
> EmbeddedCache.getGroup(G). This is something useful for us in a broader 
> context, as it is the step needed to be able to deprecated AtomicMaps and get 
> suggest them being replaced with Grouping. 
> 
> This approach still has some limitations compared to the current embedded 
> integration: 
> - performance caused by the lack of transactions: this means increased TCP 
> chattiness between the Hot Rod client and the server. 
> - you'd have to handle atomicity, potentially by retrying an operation
> 
> What do you think?
> 
> 
> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
> 
>> 
>> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  
>> wrote:
>> 
>>> It's an interesting approach that would work fine-ish for entities
>>> assuming the Hot Rod client is multi threaded and assuming the client
>>> uses Future to parallelize the calls.
>> 
>> The Java Hotrod client is both multithreaded and exposes an Async API.
>> 
>>> 
>>> But it won't work for associations as we have them designed today.
>>> Each association - or more precisely the query results to go from an
>>> entity A1 to the list of entities B associated to it - is represented by
>>> an AtomicMap.
>>> Each entry in this map does correspond to an entry in the association.
>>> 
>>> While we can "guess" the column names and build from the metadata the
>>> list of composed keys for entities, we cannot do the same for
>>> associations as the key is literally the (composite) id of the
>>> association and we cannot guess that most of the time (we can in very
>>> pathological cases).
>>> We could imagine that we list the association row keys in a special
>>> entry to work around that but this approach is just as problematic and
>>> is conceptually the same.
>>> The only solution would be to lock the whole association for each
>>> operation and I guess impose some versioning / optimistic lock.
>>> 
>>> That is not a pattern that scales sufficiently from my experience.
>> 
>> I think so too :-)
>> 
>>> That's the problem with interconnected data :)
>>> 
>>> Emmanuel
>>> 
>>> On Mon 2013-11-18 23:05, Mircea Markus wrote:
 Neither the grouping API nor the AtomicMap work over hotrod.
 Between the grouping API and AtomicMap, I think the one that would make 
 more sense migrating is the grouping API.
 One way or the other, I think the hotrod protocol would require an 
 enhancement - mind raising a JIRA for that?
 For now I guess you can sacrifice performance and always sending the 
 entire object across on every update instead of only the deltas?
 
 On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
 wrote:
 
> Someone mentioned the grouping API as some sort of alternative to
> AtomicMap. Maybe we should use that?
> Note that if we don't have a fine-grained approach we will need to
> make sure we *copy* the complex data structure upon reads to mimic
> proper transaction isolation.
> 
> On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>> On 12 November 2013 14:54, Emmanuel Bernard  
>> wrote:
>>> On the transaction side, we can start without them.
>> 
>> +1 on omitting transactions for now.
>> 
>>

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

On Jan 21, 2014, at 4:08 PM, Sanne Grinovero  wrote:

> Hi Mircea,
> could you explain how Grouping is different than AtomicMaps ?

Here's the original thread where this has been discussed: http://goo.gl/WNs6KY 
I would add to that that the AtomicMap requires transactions, which grouping 
doesn't. Also in the context of hotrod (i.e. this email thread) FGAM is a 
structure that would be harder to migrate over 

> I understand you're all suggesting to move to AtomicMaps as "the
> implementation is better"

we're suggesting to move from the AM to grouping

> but is that an implementation detail, or how
> is it inherently different so that we can build something more
> reliable on it?

They both are doing pretty much the same thing, so it's more a matter of 
choosing one instead of the other. Grouping fits way nicer into the picture, 
both as a concept and the  implementation.

> 
>> From the limited knowledge I have in this area, I have been assuming -
> since they have very similar properties - that this was essentially a
> different syntax to get to the same semantics but obviously I'm wrong.
> 
> It would be especially helpfull to have a clear comparison on the
> different semantics in terms of transactions, atomicity and visibility
> of state across the three kinds: AtomicMaps, FineGrainedAtomicMaps,
> Grouping.
> 
> Let's also keep in mind that Hibernate OGM uses a carefully selected
> combination of *both* AtomicMap and FGAM instances - depending on the
> desired semantics we want to achieve, so since those two where clearly
> different and we actually build on those differences - I'm not seeing
> how we could migrate two different things to the same construct
> without having to move "fishy locking details" out of Infinispan but
> in OGM, and I wouldn't be too happy with that as such logic would
> belong in Infinispan to provide.

I wasn't aware that OGM still uses AtomicMap, but the only case in which I 
imagine that would be useful is in order to force a lock on the whole 
AtomicMap. Is that so or some other aspect that I'm missing? 

> 
> - Sanne
> 
> 
> On 21 January 2014 15:07, Mircea Markus  wrote:
>> Hi Emmanuel,
>> 
>> Just had a good chat with Davide on this and one solution to overcome the 
>> shortcoming you mentioned in the above email would be to enhance the hotrod 
>> client to support grouping:
>> 
>> RemoteClient.put(G g, K k, V v); //first param is the group
>> RemoteClinet.getGroup(G g) : Map;
>> 
>> It requires an enhancement on our local grouping API: 
>> EmbeddedCache.getGroup(G). This is something useful for us in a broader 
>> context, as it is the step needed to be able to deprecated AtomicMaps and 
>> get suggest them being replaced with Grouping.
>> 
>> This approach still has some limitations compared to the current embedded 
>> integration:
>> - performance caused by the lack of transactions: this means increased TCP 
>> chattiness between the Hot Rod client and the server.
>> - you'd have to handle atomicity, potentially by retrying an operation
>> 
>> What do you think?
>> 
>> 
>> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
>> 
>>> 
>>> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  
>>> wrote:
>>> 
 It's an interesting approach that would work fine-ish for entities
 assuming the Hot Rod client is multi threaded and assuming the client
 uses Future to parallelize the calls.
>>> 
>>> The Java Hotrod client is both multithreaded and exposes an Async API.
>>> 
 
 But it won't work for associations as we have them designed today.
 Each association - or more precisely the query results to go from an
 entity A1 to the list of entities B associated to it - is represented by
 an AtomicMap.
 Each entry in this map does correspond to an entry in the association.
 
 While we can "guess" the column names and build from the metadata the
 list of composed keys for entities, we cannot do the same for
 associations as the key is literally the (composite) id of the
 association and we cannot guess that most of the time (we can in very
 pathological cases).
 We could imagine that we list the association row keys in a special
 entry to work around that but this approach is just as problematic and
 is conceptually the same.
 The only solution would be to lock the whole association for each
 operation and I guess impose some versioning / optimistic lock.
 
 That is not a pattern that scales sufficiently from my experience.
>>> 
>>> I think so too :-)
>>> 
 That's the problem with interconnected data :)
 
 Emmanuel
 
 On Mon 2013-11-18 23:05, Mircea Markus wrote:
> Neither the grouping API nor the AtomicMap work over hotrod.
> Between the grouping API and AtomicMap, I think the one that would make 
> more sense migrating is the grouping API.
> One way or the other, I think the hotrod protocol would require an 
> enhancement - mind raising a JIRA for that?
>

Re: [infinispan-dev] Integration between HotRod and OGM

[Sanne Grinovero]

Hi Mircea,
could you explain how Grouping is different than AtomicMaps ?
I understand you're all suggesting to move to AtomicMaps as "the
implementation is better" but is that an implementation detail, or how
is it inherently different so that we can build something more
reliable on it?

>From the limited knowledge I have in this area, I have been assuming -
since they have very similar properties - that this was essentially a
different syntax to get to the same semantics but obviously I'm wrong.

It would be especially helpfull to have a clear comparison on the
different semantics in terms of transactions, atomicity and visibility
of state across the three kinds: AtomicMaps, FineGrainedAtomicMaps,
Grouping.

Let's also keep in mind that Hibernate OGM uses a carefully selected
combination of *both* AtomicMap and FGAM instances - depending on the
desired semantics we want to achieve, so since those two where clearly
different and we actually build on those differences - I'm not seeing
how we could migrate two different things to the same construct
without having to move "fishy locking details" out of Infinispan but
in OGM, and I wouldn't be too happy with that as such logic would
belong in Infinispan to provide.

- Sanne


On 21 January 2014 15:07, Mircea Markus  wrote:
> Hi Emmanuel,
>
> Just had a good chat with Davide on this and one solution to overcome the 
> shortcoming you mentioned in the above email would be to enhance the hotrod 
> client to support grouping:
>
> RemoteClient.put(G g, K k, V v); //first param is the group
> RemoteClinet.getGroup(G g) : Map;
>
> It requires an enhancement on our local grouping API: 
> EmbeddedCache.getGroup(G). This is something useful for us in a broader 
> context, as it is the step needed to be able to deprecated AtomicMaps and get 
> suggest them being replaced with Grouping.
>
> This approach still has some limitations compared to the current embedded 
> integration:
> - performance caused by the lack of transactions: this means increased TCP 
> chattiness between the Hot Rod client and the server.
> - you'd have to handle atomicity, potentially by retrying an operation
>
> What do you think?
>
>
> On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:
>
>>
>> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  
>> wrote:
>>
>>> It's an interesting approach that would work fine-ish for entities
>>> assuming the Hot Rod client is multi threaded and assuming the client
>>> uses Future to parallelize the calls.
>>
>> The Java Hotrod client is both multithreaded and exposes an Async API.
>>
>>>
>>> But it won't work for associations as we have them designed today.
>>> Each association - or more precisely the query results to go from an
>>> entity A1 to the list of entities B associated to it - is represented by
>>> an AtomicMap.
>>> Each entry in this map does correspond to an entry in the association.
>>>
>>> While we can "guess" the column names and build from the metadata the
>>> list of composed keys for entities, we cannot do the same for
>>> associations as the key is literally the (composite) id of the
>>> association and we cannot guess that most of the time (we can in very
>>> pathological cases).
>>> We could imagine that we list the association row keys in a special
>>> entry to work around that but this approach is just as problematic and
>>> is conceptually the same.
>>> The only solution would be to lock the whole association for each
>>> operation and I guess impose some versioning / optimistic lock.
>>>
>>> That is not a pattern that scales sufficiently from my experience.
>>
>> I think so too :-)
>>
>>> That's the problem with interconnected data :)
>>>
>>> Emmanuel
>>>
>>> On Mon 2013-11-18 23:05, Mircea Markus wrote:
 Neither the grouping API nor the AtomicMap work over hotrod.
 Between the grouping API and AtomicMap, I think the one that would make 
 more sense migrating is the grouping API.
 One way or the other, I think the hotrod protocol would require an 
 enhancement - mind raising a JIRA for that?
 For now I guess you can sacrifice performance and always sending the 
 entire object across on every update instead of only the deltas?

 On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
 wrote:

> Someone mentioned the grouping API as some sort of alternative to
> AtomicMap. Maybe we should use that?
> Note that if we don't have a fine-grained approach we will need to
> make sure we *copy* the complex data structure upon reads to mimic
> proper transaction isolation.
>
> On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>> On 12 November 2013 14:54, Emmanuel Bernard  
>> wrote:
>>> On the transaction side, we can start without them.
>>
>> +1 on omitting transactions for now.
>>
>> And on the missing AtomicMaps, I hope the Infinispan will want to 
>> implement it?
>> Would be good to eventually converge on similar featuresets on remote
>> vs embe

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

Hi Emmanuel,

Just had a good chat with Davide on this and one solution to overcome the 
shortcoming you mentioned in the above email would be to enhance the hotrod 
client to support grouping:

RemoteClient.put(G g, K k, V v); //first param is the group
RemoteClinet.getGroup(G g) : Map;

It requires an enhancement on our local grouping API: 
EmbeddedCache.getGroup(G). This is something useful for us in a broader 
context, as it is the step needed to be able to deprecated AtomicMaps and get 
suggest them being replaced with Grouping. 

This approach still has some limitations compared to the current embedded 
integration: 
- performance caused by the lack of transactions: this means increased TCP 
chattiness between the Hot Rod client and the server. 
- you'd have to handle atomicity, potentially by retrying an operation

What do you think?


On Dec 3, 2013, at 3:10 AM, Mircea Markus  wrote:

> 
> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  wrote:
> 
>> It's an interesting approach that would work fine-ish for entities
>> assuming the Hot Rod client is multi threaded and assuming the client
>> uses Future to parallelize the calls.
> 
> The Java Hotrod client is both multithreaded and exposes an Async API.
> 
>> 
>> But it won't work for associations as we have them designed today.
>> Each association - or more precisely the query results to go from an
>> entity A1 to the list of entities B associated to it - is represented by
>> an AtomicMap.
>> Each entry in this map does correspond to an entry in the association.
>> 
>> While we can "guess" the column names and build from the metadata the
>> list of composed keys for entities, we cannot do the same for
>> associations as the key is literally the (composite) id of the
>> association and we cannot guess that most of the time (we can in very
>> pathological cases).
>> We could imagine that we list the association row keys in a special
>> entry to work around that but this approach is just as problematic and
>> is conceptually the same.
>> The only solution would be to lock the whole association for each
>> operation and I guess impose some versioning / optimistic lock.
>> 
>> That is not a pattern that scales sufficiently from my experience.
> 
> I think so too :-)
> 
>> That's the problem with interconnected data :)
>> 
>> Emmanuel
>> 
>> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>>> Neither the grouping API nor the AtomicMap work over hotrod.
>>> Between the grouping API and AtomicMap, I think the one that would make 
>>> more sense migrating is the grouping API.
>>> One way or the other, I think the hotrod protocol would require an 
>>> enhancement - mind raising a JIRA for that?
>>> For now I guess you can sacrifice performance and always sending the entire 
>>> object across on every update instead of only the deltas?
>>> 
>>> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
>>> wrote:
>>> 
 Someone mentioned the grouping API as some sort of alternative to
 AtomicMap. Maybe we should use that?
 Note that if we don't have a fine-grained approach we will need to
 make sure we *copy* the complex data structure upon reads to mimic
 proper transaction isolation.
 
 On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
> On 12 November 2013 14:54, Emmanuel Bernard  
> wrote:
>> On the transaction side, we can start without them.
> 
> +1 on omitting transactions for now.
> 
> And on the missing AtomicMaps, I hope the Infinispan will want to 
> implement it?
> Would be good to eventually converge on similar featuresets on remote
> vs embedded APIs.
> 
> I know the embedded version relies on batching/transactions, but I
> guess we could obtain a similar effect with some ad-hoc commands in
> Hot Rod?
> 
> Sanne
> 
>> 
>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>>> Hi,
>>> I'm working on the integration between HotRod and OGM.
>>> 
>>> We already have a dialect for Inifinispan and I'm trying to follow the 
>>> same
>>> logic.
>>> At the moment I'm having two problems:
>>> 
>>> 1) In the Infinispan dialect we are using the AtomicMap and the
>>> AtomicMapLookup but this classes don't work with the RemoteCache. Is 
>>> there
>>> an equivalent for HotRod?
>>> 
>>> 2) As far as I know HotRod does not support transactions. I've found a 
>>> link
>>> to a branch on Mircea repository:
>>> https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>>> Is this something I could/should use?
>>> 
>>> Any help is appreciated.
>>> 
>>> Thanks,
>>> Davide
>> 
>>> ___
>>> infinispan-dev mailing list
>>> infinispan-dev@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard  wrote:

> It's an interesting approach that would work fine-ish for entities
> assuming the Hot Rod client is multi threaded and assuming the client
> uses Future to parallelize the calls.

The Java Hotrod client is both multithreaded and exposes an Async API.

> 
> But it won't work for associations as we have them designed today.
> Each association - or more precisely the query results to go from an
> entity A1 to the list of entities B associated to it - is represented by
> an AtomicMap.
> Each entry in this map does correspond to an entry in the association.
> 
> While we can "guess" the column names and build from the metadata the
> list of composed keys for entities, we cannot do the same for
> associations as the key is literally the (composite) id of the
> association and we cannot guess that most of the time (we can in very
> pathological cases).
> We could imagine that we list the association row keys in a special
> entry to work around that but this approach is just as problematic and
> is conceptually the same.
> The only solution would be to lock the whole association for each
> operation and I guess impose some versioning / optimistic lock.
> 
> That is not a pattern that scales sufficiently from my experience.

I think so too :-)

> That's the problem with interconnected data :)
> 
> Emmanuel
> 
> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>> Neither the grouping API nor the AtomicMap work over hotrod.
>> Between the grouping API and AtomicMap, I think the one that would make more 
>> sense migrating is the grouping API.
>> One way or the other, I think the hotrod protocol would require an 
>> enhancement - mind raising a JIRA for that?
>> For now I guess you can sacrifice performance and always sending the entire 
>> object across on every update instead of only the deltas?
>> 
>> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  wrote:
>> 
>>> Someone mentioned the grouping API as some sort of alternative to
>>> AtomicMap. Maybe we should use that?
>>> Note that if we don't have a fine-grained approach we will need to
>>> make sure we *copy* the complex data structure upon reads to mimic
>>> proper transaction isolation.
>>> 
>>> On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
 On 12 November 2013 14:54, Emmanuel Bernard  wrote:
> On the transaction side, we can start without them.
 
 +1 on omitting transactions for now.
 
 And on the missing AtomicMaps, I hope the Infinispan will want to 
 implement it?
 Would be good to eventually converge on similar featuresets on remote
 vs embedded APIs.
 
 I know the embedded version relies on batching/transactions, but I
 guess we could obtain a similar effect with some ad-hoc commands in
 Hot Rod?
 
 Sanne
 
> 
> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>> Hi,
>> I'm working on the integration between HotRod and OGM.
>> 
>> We already have a dialect for Inifinispan and I'm trying to follow the 
>> same
>> logic.
>> At the moment I'm having two problems:
>> 
>> 1) In the Infinispan dialect we are using the AtomicMap and the
>> AtomicMapLookup but this classes don't work with the RemoteCache. Is 
>> there
>> an equivalent for HotRod?
>> 
>> 2) As far as I know HotRod does not support transactions. I've found a 
>> link
>> to a branch on Mircea repository:
>> https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>> Is this something I could/should use?
>> 
>> Any help is appreciated.
>> 
>> Thanks,
>> Davide
> 
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
 ___
 infinispan-dev mailing list
 infinispan-dev@lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> ___
>>> infinispan-dev mailing list
>>> infinispan-dev@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> Cheers,
>> -- 
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>> 
>> 
>> 
>> 
>> 
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





___
infinispan-dev mailing list
infin

Re: [infinispan-dev] Integration between HotRod and OGM

[Sanne Grinovero]

On 19 November 2013 12:56, Emmanuel Bernard  wrote:
> We could if someone puts a gun to my head.
> But that would be the *only* backend that has to rely on query for its
> association.  Hibernate OGM has a strong design bias towards totally
> controlling how CRUD is done to guarantee the consistency. If you
> introduce a search engine, all bets are off.

I think it's something to consider as that's what Hibernate ORM does
(when targeting relational databases), so we ease the mismatch.
The issues you've opened are probably preferable, but remote queries
are the only viable option I'm seeing which is available now.

Sanne

>
> On Tue 2013-11-19 10:41, Sanne Grinovero wrote:
>> We could rely on remote queries?
>>
>> On 19 November 2013 10:31, Emmanuel Bernard  wrote:
>> > This message was a response to a different email discussing the composite
>> > key approach "k1 - a1" "k1 - a2".
>> >
>> > The actual JIRAs are opened :
>> >
>> > * https://issues.jboss.org/browse/ISPN-3732
>> > * https://issues.jboss.org/browse/ISPN-3733
>> >
>> > Emmanuel
>> >
>> > On Tue 2013-11-19 11:22, Emmanuel Bernard wrote:
>> >> It's an interesting approach that would work fine-ish for entities
>> >> assuming the Hot Rod client is multi threaded and assuming the client
>> >> uses Future to parallelize the calls.
>> >>
>> >> But it won't work for associations as we have them designed today.
>> >> Each association - or more precisely the query results to go from an
>> >> entity A1 to the list of entities B associated to it - is represented by
>> >> an AtomicMap.
>> >> Each entry in this map does correspond to an entry in the association.
>> >>
>> >> While we can "guess" the column names and build from the metadata the
>> >> list of composed keys for entities, we cannot do the same for
>> >> associations as the key is literally the (composite) id of the
>> >> association and we cannot guess that most of the time (we can in very
>> >> pathological cases).
>> >> We could imagine that we list the association row keys in a special
>> >> entry to work around that but this approach is just as problematic and
>> >> is conceptually the same.
>> >> The only solution would be to lock the whole association for each
>> >> operation and I guess impose some versioning / optimistic lock.
>> >>
>> >> That is not a pattern that scales sufficiently from my experience.
>> >> That's the problem with interconnected data :)
>> >>
>> >> Emmanuel
>> >>
>> >> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>> >> > Neither the grouping API nor the AtomicMap work over hotrod.
>> >> > Between the grouping API and AtomicMap, I think the one that would make 
>> >> > more sense migrating is the grouping API.
>> >> > One way or the other, I think the hotrod protocol would require an 
>> >> > enhancement - mind raising a JIRA for that?
>> >> > For now I guess you can sacrifice performance and always sending the 
>> >> > entire object across on every update instead of only the deltas?
>> >> >
>> >> > On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
>> >> > wrote:
>> >> >
>> >> > > Someone mentioned the grouping API as some sort of alternative to
>> >> > > AtomicMap. Maybe we should use that?
>> >> > > Note that if we don't have a fine-grained approach we will need to
>> >> > > make sure we *copy* the complex data structure upon reads to mimic
>> >> > > proper transaction isolation.
>> >> > >
>> >> > > On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>> >> > >> On 12 November 2013 14:54, Emmanuel Bernard  
>> >> > >> wrote:
>> >> > >>> On the transaction side, we can start without them.
>> >> > >>
>> >> > >> +1 on omitting transactions for now.
>> >> > >>
>> >> > >> And on the missing AtomicMaps, I hope the Infinispan will want to 
>> >> > >> implement it?
>> >> > >> Would be good to eventually converge on similar featuresets on remote
>> >> > >> vs embedded APIs.
>> >> > >>
>> >> > >> I know the embedded version relies on batching/transactions, but I
>> >> > >> guess we could obtain a similar effect with some ad-hoc commands in
>> >> > >> Hot Rod?
>> >> > >>
>> >> > >> Sanne
>> >> > >>
>> >> > >>>
>> >> > >>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>> >> >  Hi,
>> >> >  I'm working on the integration between HotRod and OGM.
>> >> > 
>> >> >  We already have a dialect for Inifinispan and I'm trying to follow 
>> >> >  the same
>> >> >  logic.
>> >> >  At the moment I'm having two problems:
>> >> > 
>> >> >  1) In the Infinispan dialect we are using the AtomicMap and the
>> >> >  AtomicMapLookup but this classes don't work with the RemoteCache. 
>> >> >  Is there
>> >> >  an equivalent for HotRod?
>> >> > 
>> >> >  2) As far as I know HotRod does not support transactions. I've 
>> >> >  found a link
>> >> >  to a branch on Mircea repository:
>> >> >  https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>> >> >  Is this something I could/should use?
>> >> > 
>> >> >  Any help is appre

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

We could if someone puts a gun to my head.
But that would be the *only* backend that has to rely on query for its
association.  Hibernate OGM has a strong design bias towards totally
controlling how CRUD is done to guarantee the consistency. If you
introduce a search engine, all bets are off.

On Tue 2013-11-19 10:41, Sanne Grinovero wrote:
> We could rely on remote queries?
> 
> On 19 November 2013 10:31, Emmanuel Bernard  wrote:
> > This message was a response to a different email discussing the composite
> > key approach "k1 - a1" "k1 - a2".
> >
> > The actual JIRAs are opened :
> >
> > * https://issues.jboss.org/browse/ISPN-3732
> > * https://issues.jboss.org/browse/ISPN-3733
> >
> > Emmanuel
> >
> > On Tue 2013-11-19 11:22, Emmanuel Bernard wrote:
> >> It's an interesting approach that would work fine-ish for entities
> >> assuming the Hot Rod client is multi threaded and assuming the client
> >> uses Future to parallelize the calls.
> >>
> >> But it won't work for associations as we have them designed today.
> >> Each association - or more precisely the query results to go from an
> >> entity A1 to the list of entities B associated to it - is represented by
> >> an AtomicMap.
> >> Each entry in this map does correspond to an entry in the association.
> >>
> >> While we can "guess" the column names and build from the metadata the
> >> list of composed keys for entities, we cannot do the same for
> >> associations as the key is literally the (composite) id of the
> >> association and we cannot guess that most of the time (we can in very
> >> pathological cases).
> >> We could imagine that we list the association row keys in a special
> >> entry to work around that but this approach is just as problematic and
> >> is conceptually the same.
> >> The only solution would be to lock the whole association for each
> >> operation and I guess impose some versioning / optimistic lock.
> >>
> >> That is not a pattern that scales sufficiently from my experience.
> >> That's the problem with interconnected data :)
> >>
> >> Emmanuel
> >>
> >> On Mon 2013-11-18 23:05, Mircea Markus wrote:
> >> > Neither the grouping API nor the AtomicMap work over hotrod.
> >> > Between the grouping API and AtomicMap, I think the one that would make 
> >> > more sense migrating is the grouping API.
> >> > One way or the other, I think the hotrod protocol would require an 
> >> > enhancement - mind raising a JIRA for that?
> >> > For now I guess you can sacrifice performance and always sending the 
> >> > entire object across on every update instead of only the deltas?
> >> >
> >> > On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
> >> > wrote:
> >> >
> >> > > Someone mentioned the grouping API as some sort of alternative to
> >> > > AtomicMap. Maybe we should use that?
> >> > > Note that if we don't have a fine-grained approach we will need to
> >> > > make sure we *copy* the complex data structure upon reads to mimic
> >> > > proper transaction isolation.
> >> > >
> >> > > On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
> >> > >> On 12 November 2013 14:54, Emmanuel Bernard  
> >> > >> wrote:
> >> > >>> On the transaction side, we can start without them.
> >> > >>
> >> > >> +1 on omitting transactions for now.
> >> > >>
> >> > >> And on the missing AtomicMaps, I hope the Infinispan will want to 
> >> > >> implement it?
> >> > >> Would be good to eventually converge on similar featuresets on remote
> >> > >> vs embedded APIs.
> >> > >>
> >> > >> I know the embedded version relies on batching/transactions, but I
> >> > >> guess we could obtain a similar effect with some ad-hoc commands in
> >> > >> Hot Rod?
> >> > >>
> >> > >> Sanne
> >> > >>
> >> > >>>
> >> > >>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
> >> >  Hi,
> >> >  I'm working on the integration between HotRod and OGM.
> >> > 
> >> >  We already have a dialect for Inifinispan and I'm trying to follow 
> >> >  the same
> >> >  logic.
> >> >  At the moment I'm having two problems:
> >> > 
> >> >  1) In the Infinispan dialect we are using the AtomicMap and the
> >> >  AtomicMapLookup but this classes don't work with the RemoteCache. 
> >> >  Is there
> >> >  an equivalent for HotRod?
> >> > 
> >> >  2) As far as I know HotRod does not support transactions. I've 
> >> >  found a link
> >> >  to a branch on Mircea repository:
> >> >  https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
> >> >  Is this something I could/should use?
> >> > 
> >> >  Any help is appreciated.
> >> > 
> >> >  Thanks,
> >> >  Davide
> >> > >>>
> >> >  ___
> >> >  infinispan-dev mailing list
> >> >  infinispan-dev@lists.jboss.org
> >> >  https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> > >>>
> >> > >>> ___
> >> > >>> infinispan-dev mailing list
> >> > >>> infinispan-dev@lists.jboss.org
> >

Re: [infinispan-dev] Integration between HotRod and OGM

[Sanne Grinovero]

We could rely on remote queries?

On 19 November 2013 10:31, Emmanuel Bernard  wrote:
> This message was a response to a different email discussing the composite
> key approach "k1 - a1" "k1 - a2".
>
> The actual JIRAs are opened :
>
> * https://issues.jboss.org/browse/ISPN-3732
> * https://issues.jboss.org/browse/ISPN-3733
>
> Emmanuel
>
> On Tue 2013-11-19 11:22, Emmanuel Bernard wrote:
>> It's an interesting approach that would work fine-ish for entities
>> assuming the Hot Rod client is multi threaded and assuming the client
>> uses Future to parallelize the calls.
>>
>> But it won't work for associations as we have them designed today.
>> Each association - or more precisely the query results to go from an
>> entity A1 to the list of entities B associated to it - is represented by
>> an AtomicMap.
>> Each entry in this map does correspond to an entry in the association.
>>
>> While we can "guess" the column names and build from the metadata the
>> list of composed keys for entities, we cannot do the same for
>> associations as the key is literally the (composite) id of the
>> association and we cannot guess that most of the time (we can in very
>> pathological cases).
>> We could imagine that we list the association row keys in a special
>> entry to work around that but this approach is just as problematic and
>> is conceptually the same.
>> The only solution would be to lock the whole association for each
>> operation and I guess impose some versioning / optimistic lock.
>>
>> That is not a pattern that scales sufficiently from my experience.
>> That's the problem with interconnected data :)
>>
>> Emmanuel
>>
>> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>> > Neither the grouping API nor the AtomicMap work over hotrod.
>> > Between the grouping API and AtomicMap, I think the one that would make 
>> > more sense migrating is the grouping API.
>> > One way or the other, I think the hotrod protocol would require an 
>> > enhancement - mind raising a JIRA for that?
>> > For now I guess you can sacrifice performance and always sending the 
>> > entire object across on every update instead of only the deltas?
>> >
>> > On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
>> > wrote:
>> >
>> > > Someone mentioned the grouping API as some sort of alternative to
>> > > AtomicMap. Maybe we should use that?
>> > > Note that if we don't have a fine-grained approach we will need to
>> > > make sure we *copy* the complex data structure upon reads to mimic
>> > > proper transaction isolation.
>> > >
>> > > On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>> > >> On 12 November 2013 14:54, Emmanuel Bernard  
>> > >> wrote:
>> > >>> On the transaction side, we can start without them.
>> > >>
>> > >> +1 on omitting transactions for now.
>> > >>
>> > >> And on the missing AtomicMaps, I hope the Infinispan will want to 
>> > >> implement it?
>> > >> Would be good to eventually converge on similar featuresets on remote
>> > >> vs embedded APIs.
>> > >>
>> > >> I know the embedded version relies on batching/transactions, but I
>> > >> guess we could obtain a similar effect with some ad-hoc commands in
>> > >> Hot Rod?
>> > >>
>> > >> Sanne
>> > >>
>> > >>>
>> > >>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>> >  Hi,
>> >  I'm working on the integration between HotRod and OGM.
>> > 
>> >  We already have a dialect for Inifinispan and I'm trying to follow 
>> >  the same
>> >  logic.
>> >  At the moment I'm having two problems:
>> > 
>> >  1) In the Infinispan dialect we are using the AtomicMap and the
>> >  AtomicMapLookup but this classes don't work with the RemoteCache. Is 
>> >  there
>> >  an equivalent for HotRod?
>> > 
>> >  2) As far as I know HotRod does not support transactions. I've found 
>> >  a link
>> >  to a branch on Mircea repository:
>> >  https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>> >  Is this something I could/should use?
>> > 
>> >  Any help is appreciated.
>> > 
>> >  Thanks,
>> >  Davide
>> > >>>
>> >  ___
>> >  infinispan-dev mailing list
>> >  infinispan-dev@lists.jboss.org
>> >  https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> > >>>
>> > >>> ___
>> > >>> infinispan-dev mailing list
>> > >>> infinispan-dev@lists.jboss.org
>> > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> > >> ___
>> > >> infinispan-dev mailing list
>> > >> infinispan-dev@lists.jboss.org
>> > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> > > ___
>> > > infinispan-dev mailing list
>> > > infinispan-dev@lists.jboss.org
>> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >
>> > Cheers,
>> > --
>> > Mircea Markus
>> > Infinispan lead (www.infinispan.org)
>> >
>> >
>> >
>> >
>> >
>> 

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

This message was a response to a different email discussing the composite
key approach "k1 - a1" "k1 - a2".

The actual JIRAs are opened :

* https://issues.jboss.org/browse/ISPN-3732
* https://issues.jboss.org/browse/ISPN-3733

Emmanuel

On Tue 2013-11-19 11:22, Emmanuel Bernard wrote:
> It's an interesting approach that would work fine-ish for entities
> assuming the Hot Rod client is multi threaded and assuming the client
> uses Future to parallelize the calls.
> 
> But it won't work for associations as we have them designed today.
> Each association - or more precisely the query results to go from an
> entity A1 to the list of entities B associated to it - is represented by
> an AtomicMap.
> Each entry in this map does correspond to an entry in the association.
> 
> While we can "guess" the column names and build from the metadata the
> list of composed keys for entities, we cannot do the same for
> associations as the key is literally the (composite) id of the
> association and we cannot guess that most of the time (we can in very
> pathological cases).
> We could imagine that we list the association row keys in a special
> entry to work around that but this approach is just as problematic and
> is conceptually the same.
> The only solution would be to lock the whole association for each
> operation and I guess impose some versioning / optimistic lock.
> 
> That is not a pattern that scales sufficiently from my experience.
> That's the problem with interconnected data :)
> 
> Emmanuel
> 
> On Mon 2013-11-18 23:05, Mircea Markus wrote:
> > Neither the grouping API nor the AtomicMap work over hotrod.
> > Between the grouping API and AtomicMap, I think the one that would make 
> > more sense migrating is the grouping API.
> > One way or the other, I think the hotrod protocol would require an 
> > enhancement - mind raising a JIRA for that?
> > For now I guess you can sacrifice performance and always sending the entire 
> > object across on every update instead of only the deltas?
> > 
> > On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  
> > wrote:
> > 
> > > Someone mentioned the grouping API as some sort of alternative to
> > > AtomicMap. Maybe we should use that?
> > > Note that if we don't have a fine-grained approach we will need to
> > > make sure we *copy* the complex data structure upon reads to mimic
> > > proper transaction isolation.
> > > 
> > > On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
> > >> On 12 November 2013 14:54, Emmanuel Bernard  
> > >> wrote:
> > >>> On the transaction side, we can start without them.
> > >> 
> > >> +1 on omitting transactions for now.
> > >> 
> > >> And on the missing AtomicMaps, I hope the Infinispan will want to 
> > >> implement it?
> > >> Would be good to eventually converge on similar featuresets on remote
> > >> vs embedded APIs.
> > >> 
> > >> I know the embedded version relies on batching/transactions, but I
> > >> guess we could obtain a similar effect with some ad-hoc commands in
> > >> Hot Rod?
> > >> 
> > >> Sanne
> > >> 
> > >>> 
> > >>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
> >  Hi,
> >  I'm working on the integration between HotRod and OGM.
> >  
> >  We already have a dialect for Inifinispan and I'm trying to follow the 
> >  same
> >  logic.
> >  At the moment I'm having two problems:
> >  
> >  1) In the Infinispan dialect we are using the AtomicMap and the
> >  AtomicMapLookup but this classes don't work with the RemoteCache. Is 
> >  there
> >  an equivalent for HotRod?
> >  
> >  2) As far as I know HotRod does not support transactions. I've found a 
> >  link
> >  to a branch on Mircea repository:
> >  https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
> >  Is this something I could/should use?
> >  
> >  Any help is appreciated.
> >  
> >  Thanks,
> >  Davide
> > >>> 
> >  ___
> >  infinispan-dev mailing list
> >  infinispan-dev@lists.jboss.org
> >  https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >>> 
> > >>> ___
> > >>> infinispan-dev mailing list
> > >>> infinispan-dev@lists.jboss.org
> > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >> ___
> > >> infinispan-dev mailing list
> > >> infinispan-dev@lists.jboss.org
> > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > > ___
> > > infinispan-dev mailing list
> > > infinispan-dev@lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > 
> > Cheers,
> > -- 
> > Mircea Markus
> > Infinispan lead (www.infinispan.org)
> > 
> > 
> > 
> > 
> > 
> > ___
> > infinispan-dev mailing list
> > infinispan-dev@lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

It's an interesting approach that would work fine-ish for entities
assuming the Hot Rod client is multi threaded and assuming the client
uses Future to parallelize the calls.

But it won't work for associations as we have them designed today.
Each association - or more precisely the query results to go from an
entity A1 to the list of entities B associated to it - is represented by
an AtomicMap.
Each entry in this map does correspond to an entry in the association.

While we can "guess" the column names and build from the metadata the
list of composed keys for entities, we cannot do the same for
associations as the key is literally the (composite) id of the
association and we cannot guess that most of the time (we can in very
pathological cases).
We could imagine that we list the association row keys in a special
entry to work around that but this approach is just as problematic and
is conceptually the same.
The only solution would be to lock the whole association for each
operation and I guess impose some versioning / optimistic lock.

That is not a pattern that scales sufficiently from my experience.
That's the problem with interconnected data :)

Emmanuel

On Mon 2013-11-18 23:05, Mircea Markus wrote:
> Neither the grouping API nor the AtomicMap work over hotrod.
> Between the grouping API and AtomicMap, I think the one that would make more 
> sense migrating is the grouping API.
> One way or the other, I think the hotrod protocol would require an 
> enhancement - mind raising a JIRA for that?
> For now I guess you can sacrifice performance and always sending the entire 
> object across on every update instead of only the deltas?
> 
> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  wrote:
> 
> > Someone mentioned the grouping API as some sort of alternative to
> > AtomicMap. Maybe we should use that?
> > Note that if we don't have a fine-grained approach we will need to
> > make sure we *copy* the complex data structure upon reads to mimic
> > proper transaction isolation.
> > 
> > On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
> >> On 12 November 2013 14:54, Emmanuel Bernard  wrote:
> >>> On the transaction side, we can start without them.
> >> 
> >> +1 on omitting transactions for now.
> >> 
> >> And on the missing AtomicMaps, I hope the Infinispan will want to 
> >> implement it?
> >> Would be good to eventually converge on similar featuresets on remote
> >> vs embedded APIs.
> >> 
> >> I know the embedded version relies on batching/transactions, but I
> >> guess we could obtain a similar effect with some ad-hoc commands in
> >> Hot Rod?
> >> 
> >> Sanne
> >> 
> >>> 
> >>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>  Hi,
>  I'm working on the integration between HotRod and OGM.
>  
>  We already have a dialect for Inifinispan and I'm trying to follow the 
>  same
>  logic.
>  At the moment I'm having two problems:
>  
>  1) In the Infinispan dialect we are using the AtomicMap and the
>  AtomicMapLookup but this classes don't work with the RemoteCache. Is 
>  there
>  an equivalent for HotRod?
>  
>  2) As far as I know HotRod does not support transactions. I've found a 
>  link
>  to a branch on Mircea repository:
>  https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>  Is this something I could/should use?
>  
>  Any help is appreciated.
>  
>  Thanks,
>  Davide
> >>> 
>  ___
>  infinispan-dev mailing list
>  infinispan-dev@lists.jboss.org
>  https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>> 
> >>> ___
> >>> infinispan-dev mailing list
> >>> infinispan-dev@lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> ___
> >> infinispan-dev mailing list
> >> infinispan-dev@lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > ___
> > infinispan-dev mailing list
> > infinispan-dev@lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> Cheers,
> -- 
> Mircea Markus
> Infinispan lead (www.infinispan.org)
> 
> 
> 
> 
> 
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

On Nov 18, 2013, at 11:52 PM, Sanne Grinovero  wrote:

>> Neither the grouping API nor the AtomicMap work over hotrod.
>> Between the grouping API and AtomicMap, I think the one that would make more 
>> sense migrating is the grouping API.
>> One way or the other, I think the hotrod protocol would require an 
>> enhancement - mind raising a JIRA for that?
>> For now I guess you can sacrifice performance and always sending the entire 
>> object across on every update instead of only the deltas?
> 
> Doesn't Hot Rod even have support for deltas?
> 
> Our main concern is not so much performance but to implement the
> correct isolation: ok we have no transactions, but still we'd be
> messing with entries which weren't even supposed to be loaded, and
> this data could get overwritten.

then why not just use a syntethic key for every key in the AtopmicMap?
e.g. if you have the following AtomicMap: (ak, {(k1, v1), (k2,v2)}) where:
ak - the key of the AtomicMap in the cache
k1, k2 - keys within the atomic map

then using some synthetic the keys: "ak_k1", "ak_k2" directly on the remote 
cache would avoid the overwriting. 

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Sanne Grinovero]

On 18 November 2013 23:05, Mircea Markus  wrote:
> Neither the grouping API nor the AtomicMap work over hotrod.
> Between the grouping API and AtomicMap, I think the one that would make more 
> sense migrating is the grouping API.
> One way or the other, I think the hotrod protocol would require an 
> enhancement - mind raising a JIRA for that?
> For now I guess you can sacrifice performance and always sending the entire 
> object across on every update instead of only the deltas?

Doesn't Hot Rod even have support for deltas?

Our main concern is not so much performance but to implement the
correct isolation: ok we have no transactions, but still we'd be
messing with entries which weren't even supposed to be loaded, and
this data could get overwritten.

Sanne

>
> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  wrote:
>
>> Someone mentioned the grouping API as some sort of alternative to
>> AtomicMap. Maybe we should use that?
>> Note that if we don't have a fine-grained approach we will need to
>> make sure we *copy* the complex data structure upon reads to mimic
>> proper transaction isolation.
>>
>> On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>>> On 12 November 2013 14:54, Emmanuel Bernard  wrote:
 On the transaction side, we can start without them.
>>>
>>> +1 on omitting transactions for now.
>>>
>>> And on the missing AtomicMaps, I hope the Infinispan will want to implement 
>>> it?
>>> Would be good to eventually converge on similar featuresets on remote
>>> vs embedded APIs.
>>>
>>> I know the embedded version relies on batching/transactions, but I
>>> guess we could obtain a similar effect with some ad-hoc commands in
>>> Hot Rod?
>>>
>>> Sanne
>>>

 On Tue 2013-11-12 14:34, Davide D'Alto wrote:
> Hi,
> I'm working on the integration between HotRod and OGM.
>
> We already have a dialect for Inifinispan and I'm trying to follow the 
> same
> logic.
> At the moment I'm having two problems:
>
> 1) In the Infinispan dialect we are using the AtomicMap and the
> AtomicMapLookup but this classes don't work with the RemoteCache. Is there
> an equivalent for HotRod?
>
> 2) As far as I know HotRod does not support transactions. I've found a 
> link
> to a branch on Mircea repository:
> https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
> Is this something I could/should use?
>
> Any help is appreciated.
>
> Thanks,
> Davide

> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

 ___
 infinispan-dev mailing list
 infinispan-dev@lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> ___
>>> infinispan-dev mailing list
>>> infinispan-dev@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Mircea Markus]

Neither the grouping API nor the AtomicMap work over hotrod.
Between the grouping API and AtomicMap, I think the one that would make more 
sense migrating is the grouping API.
One way or the other, I think the hotrod protocol would require an enhancement 
- mind raising a JIRA for that?
For now I guess you can sacrifice performance and always sending the entire 
object across on every update instead of only the deltas?

On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard  wrote:

> Someone mentioned the grouping API as some sort of alternative to
> AtomicMap. Maybe we should use that?
> Note that if we don't have a fine-grained approach we will need to
> make sure we *copy* the complex data structure upon reads to mimic
> proper transaction isolation.
> 
> On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>> On 12 November 2013 14:54, Emmanuel Bernard  wrote:
>>> On the transaction side, we can start without them.
>> 
>> +1 on omitting transactions for now.
>> 
>> And on the missing AtomicMaps, I hope the Infinispan will want to implement 
>> it?
>> Would be good to eventually converge on similar featuresets on remote
>> vs embedded APIs.
>> 
>> I know the embedded version relies on batching/transactions, but I
>> guess we could obtain a similar effect with some ad-hoc commands in
>> Hot Rod?
>> 
>> Sanne
>> 
>>> 
>>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
 Hi,
 I'm working on the integration between HotRod and OGM.
 
 We already have a dialect for Inifinispan and I'm trying to follow the same
 logic.
 At the moment I'm having two problems:
 
 1) In the Infinispan dialect we are using the AtomicMap and the
 AtomicMapLookup but this classes don't work with the RemoteCache. Is there
 an equivalent for HotRod?
 
 2) As far as I know HotRod does not support transactions. I've found a link
 to a branch on Mircea repository:
 https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
 Is this something I could/should use?
 
 Any help is appreciated.
 
 Thanks,
 Davide
>>> 
 ___
 infinispan-dev mailing list
 infinispan-dev@lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> ___
>>> infinispan-dev mailing list
>>> infinispan-dev@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

Someone mentioned the grouping API as some sort of alternative to
AtomicMap. Maybe we should use that?
Note that if we don't have a fine-grained approach we will need to
make sure we *copy* the complex data structure upon reads to mimic
proper transaction isolation.

On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
> On 12 November 2013 14:54, Emmanuel Bernard  wrote:
> > On the transaction side, we can start without them.
> 
> +1 on omitting transactions for now.
> 
> And on the missing AtomicMaps, I hope the Infinispan will want to implement 
> it?
> Would be good to eventually converge on similar featuresets on remote
> vs embedded APIs.
> 
> I know the embedded version relies on batching/transactions, but I
> guess we could obtain a similar effect with some ad-hoc commands in
> Hot Rod?
> 
> Sanne
> 
> >
> > On Tue 2013-11-12 14:34, Davide D'Alto wrote:
> >> Hi,
> >> I'm working on the integration between HotRod and OGM.
> >>
> >> We already have a dialect for Inifinispan and I'm trying to follow the same
> >> logic.
> >> At the moment I'm having two problems:
> >>
> >> 1) In the Infinispan dialect we are using the AtomicMap and the
> >> AtomicMapLookup but this classes don't work with the RemoteCache. Is there
> >> an equivalent for HotRod?
> >>
> >> 2) As far as I know HotRod does not support transactions. I've found a link
> >> to a branch on Mircea repository:
> >> https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
> >> Is this something I could/should use?
> >>
> >> Any help is appreciated.
> >>
> >> Thanks,
> >> Davide
> >
> >> ___
> >> infinispan-dev mailing list
> >> infinispan-dev@lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > ___
> > infinispan-dev mailing list
> > infinispan-dev@lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Sanne Grinovero]

On 12 November 2013 14:54, Emmanuel Bernard  wrote:
> On the transaction side, we can start without them.

+1 on omitting transactions for now.

And on the missing AtomicMaps, I hope the Infinispan will want to implement it?
Would be good to eventually converge on similar featuresets on remote
vs embedded APIs.

I know the embedded version relies on batching/transactions, but I
guess we could obtain a similar effect with some ad-hoc commands in
Hot Rod?

Sanne

>
> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>> Hi,
>> I'm working on the integration between HotRod and OGM.
>>
>> We already have a dialect for Inifinispan and I'm trying to follow the same
>> logic.
>> At the moment I'm having two problems:
>>
>> 1) In the Infinispan dialect we are using the AtomicMap and the
>> AtomicMapLookup but this classes don't work with the RemoteCache. Is there
>> an equivalent for HotRod?
>>
>> 2) As far as I know HotRod does not support transactions. I've found a link
>> to a branch on Mircea repository:
>> https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>> Is this something I could/should use?
>>
>> Any help is appreciated.
>>
>> Thanks,
>> Davide
>
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Integration between HotRod and OGM

[Emmanuel Bernard]

On the transaction side, we can start without them.

On Tue 2013-11-12 14:34, Davide D'Alto wrote:
> Hi,
> I'm working on the integration between HotRod and OGM.
> 
> We already have a dialect for Inifinispan and I'm trying to follow the same
> logic.
> At the moment I'm having two problems:
> 
> 1) In the Infinispan dialect we are using the AtomicMap and the
> AtomicMapLookup but this classes don't work with the RemoteCache. Is there
> an equivalent for HotRod?
> 
> 2) As far as I know HotRod does not support transactions. I've found a link
> to a branch on Mircea repository:
> https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
> Is this something I could/should use?
> 
> Any help is appreciated.
> 
> Thanks,
> Davide

> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev