Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-02-05 Thread Dan Berindei
On Sat, Feb 4, 2012 at 5:23 PM, Bela Ban b...@redhat.com wrote: No, Socket.send() is not a hotspot in the Java code. On the networking level, with UDP datagrams, a packet is placed into a buffer and send() returns after the packet has been placed into the buffer successfully. Some network

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-02-05 Thread Bela Ban
On 2/5/12 9:44 AM, Dan Berindei wrote: On Sat, Feb 4, 2012 at 5:23 PM, Bela Banb...@redhat.com wrote: No, Socket.send() is not a hotspot in the Java code. On the networking level, with UDP datagrams, a packet is placed into a buffer and send() returns after the packet has been placed into

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-02-05 Thread Dan Berindei
On Sun, Feb 5, 2012 at 12:24 PM, Bela Ban b...@redhat.com wrote: On 2/5/12 9:44 AM, Dan Berindei wrote: On Sat, Feb 4, 2012 at 5:23 PM, Bela Banb...@redhat.com  wrote: No, Socket.send() is not a hotspot in the Java code. On the networking level, with UDP datagrams, a packet is placed into a

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-02-05 Thread Dan Berindei
On Sun, Feb 5, 2012 at 5:56 PM, Bela Ban b...@redhat.com wrote: On 2/5/12 4:40 PM, Dan Berindei wrote: Remember that a message might be retransmitted, so it is placed into a retransmit buffer. If M1 has destination A and M2 has destination B, and we send M1 first (to A), then change M1's

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-02-04 Thread Manik Surtani
Is that a micro-optimisation? Do we know that socket.send() really is a hotspot? On 1 Feb 2012, at 00:11, Dan Berindei wrote: It's true, but then JGroups' GroupRequest does exactly the same thing... socket.send() takes some time too, I thought sending the request in parallel would mean

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-02-04 Thread Bela Ban
No, Socket.send() is not a hotspot in the Java code. On the networking level, with UDP datagrams, a packet is placed into a buffer and send() returns after the packet has been placed into the buffer successfully. Some network stacks simply discard the packet if the buffer is full, I assume.

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-31 Thread Dan Berindei
It's true, but then JGroups' GroupRequest does exactly the same thing... socket.send() takes some time too, I thought sending the request in parallel would mean calling socket.send() on a separate thread for each recipient. Cheers Dan On Fri, Jan 27, 2012 at 6:41 PM, Manik Surtani

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Manik Surtani
On 25 Jan 2012, at 08:51, Dan Berindei wrote: Slightly related, I wonder if Manik's comment is still true: if at all possible, try not to use JGroups' ANYCAST for now. Multiple (parallel) UNICASTs are much faster.) Intuitively it shouldn't be true, unicasts+FutureCollator do basically

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Manik Surtani
On 25 Jan 2012, at 09:42, Bela Ban wrote: No, parallel unicasts will be faster, as an anycast to A,B,C sends the unicasts sequentially Is this still the case in JG 3.x? -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Manik Surtani
On 25 Jan 2012, at 17:09, Dan Berindei wrote: Keep in mind that we also want to introduce eventual consistency - I think that's going to eliminate our optimization opportunity here because we'll need to get the values from a majority of owners (if not all the owners). Also keep in mind

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Manik Surtani
On 25 Jan 2012, at 17:09, Dan Berindei wrote: I think we already have a JIRA to make PutKeyValueCommands return the previous value, that would eliminate lots of GetKeyValueCommands and it would actually improve the performance of puts - we should probably make this a priority. Yes, this is

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Bela Ban
yes. On 1/27/12 12:13 PM, Manik Surtani wrote: On 25 Jan 2012, at 09:42, Bela Ban wrote: No, parallel unicasts will be faster, as an anycast to A,B,C sends the unicasts sequentially Is this still the case in JG 3.x? -- Bela Ban Lead JGroups (http://www.jgroups.org) JBoss / Red Hat

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Manik Surtani
On 25 Jan 2012, at 14:22, Mircea Markus wrote: Agreed that having a configurable remote get policy makes sense. We already have a JIRA for this[1], I'll start working on it as the performance results are hunting me. I'd like to have Dan's input on this as well first, as he has worked with

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Dan Berindei
Manik, Bela, I think we send the requests sequentially as well. In ReplicationTask.call: for (Address a : targets) { NotifyingFutureObject f = sendMessageWithFuture(constructMessage(buf, a), opts); futureCollator.watchFuture(f, a);

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-27 Thread Manik Surtani
Doesn't setBlockForResults(false) mean that we're not waiting on a response, and can proceed to the next message to the next recipient? On 27 Jan 2012, at 16:34, Dan Berindei wrote: Manik, Bela, I think we send the requests sequentially as well. In ReplicationTask.call: for

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-26 Thread Sanne Grinovero
On 26 January 2012 14:37, Mircea Markus mircea.mar...@jboss.com wrote: I'd like to have Dan's input on this as well first, as he has worked with remote gets and I still don't know why null results are not considered valid :) Pre-5.0 during state transfer an owner could return null to mean I'm

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-26 Thread Mircea Markus
On 26 Jan 2012, at 14:52, Sanne Grinovero wrote: On 26 January 2012 14:37, Mircea Markus mircea.mar...@jboss.com wrote: I'd like to have Dan's input on this as well first, as he has worked with remote gets and I still don't know why null results are not considered valid :) Pre-5.0

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Dan Berindei
Hi Sanne On Wed, Jan 25, 2012 at 1:22 AM, Sanne Grinovero sa...@infinispan.org wrote: Hello, in the method: org.infinispan.distribution.DistributionManagerImpl.retrieveFromRemoteSource(Object, InvocationContext, boolean) we have:      ListAddress targets = locate(key);      // if any of

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Bela Ban
On 1/25/12 9:51 AM, Dan Berindei wrote: Slightly related, I wonder if Manik's comment is still true: if at all possible, try not to use JGroups' ANYCAST for now. Multiple (parallel) UNICASTs are much faster.) Intuitively it shouldn't be true, unicasts+FutureCollator do basically the

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Mircea Markus
On 25 Jan 2012, at 08:51, Dan Berindei wrote: Hi Sanne On Wed, Jan 25, 2012 at 1:22 AM, Sanne Grinovero sa...@infinispan.org wrote: Hello, in the method: org.infinispan.distribution.DistributionManagerImpl.retrieveFromRemoteSource(Object, InvocationContext, boolean) we have:

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Mircea Markus
On 25 Jan 2012, at 09:42, Bela Ban wrote: On 1/25/12 9:51 AM, Dan Berindei wrote: Slightly related, I wonder if Manik's comment is still true: if at all possible, try not to use JGroups' ANYCAST for now. Multiple (parallel) UNICASTs are much faster.) Intuitively it shouldn't

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Bela Ban
On 1/25/12 12:58 PM, Mircea Markus wrote: On 25 Jan 2012, at 09:42, Bela Ban wrote: On 1/25/12 9:51 AM, Dan Berindei wrote: Slightly related, I wonder if Manik's comment is still true: if at all possible, try not to use JGroups' ANYCAST for now. Multiple (parallel) UNICASTs are

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Sanne Grinovero
On 25 January 2012 11:48, Mircea Markus mircea.mar...@jboss.com wrote: On 25 Jan 2012, at 08:51, Dan Berindei wrote: Hi Sanne On Wed, Jan 25, 2012 at 1:22 AM, Sanne Grinovero sa...@infinispan.org wrote: Hello, in the method:

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Mircea Markus
On 25 Jan 2012, at 12:06, Bela Ban wrote: On 1/25/12 12:58 PM, Mircea Markus wrote: On 25 Jan 2012, at 09:42, Bela Ban wrote: On 1/25/12 9:51 AM, Dan Berindei wrote: Slightly related, I wonder if Manik's comment is still true: if at all possible, try not to use JGroups'

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Mircea Markus
On 25 Jan 2012, at 12:06, Sanne Grinovero wrote: On 25 January 2012 11:48, Mircea Markus mircea.mar...@jboss.com wrote: On 25 Jan 2012, at 08:51, Dan Berindei wrote: Hi Sanne On Wed, Jan 25, 2012 at 1:22 AM, Sanne Grinovero sa...@infinispan.org wrote: Hello, in the method:

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Sanne Grinovero
[cut] I agree, we should not ask all replicas for the same information. Asking only one is the opposite though: I think this should be a configuration option to ask for any value between (1 and numOwner). That's because I understand it might be beneficial to ask to more than one node

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Mircea Markus
On 25 Jan 2012, at 13:25, Sanne Grinovero wrote: [cut] I agree, we should not ask all replicas for the same information. Asking only one is the opposite though: I think this should be a configuration option to ask for any value between (1 and numOwner). That's because I understand it might

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Sanne Grinovero
On 25 January 2012 13:41, Mircea Markus mircea.mar...@jboss.com wrote: On 25 Jan 2012, at 13:25, Sanne Grinovero wrote: [cut] I agree, we should not ask all replicas for the same information. Asking only one is the opposite though: I think this should be a configuration option to ask for

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Mircea Markus
One node might be busy doing GC and stay unresponsive for a whole second or longer, another one might be actually crashed and you didn't know that yet, these are unlikely but possible. All these are possible but I would rather consider them as exceptional situations, possibly handled by a

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Mircea Markus
On 25 Jan 2012, at 17:09, Dan Berindei wrote: On Wed, Jan 25, 2012 at 4:22 PM, Mircea Markus mircea.mar...@jboss.com wrote: One node might be busy doing GC and stay unresponsive for a whole second or longer, another one might be actually crashed and you didn't know that yet, these

Re: [infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-25 Thread Sanne Grinovero
On 25 January 2012 17:09, Dan Berindei dan.berin...@gmail.com wrote: On Wed, Jan 25, 2012 at 4:22 PM, Mircea Markus mircea.mar...@jboss.com wrote: One node might be busy doing GC and stay unresponsive for a whole second or longer, another one might be actually crashed and you didn't know

[infinispan-dev] DIST.retrieveFromRemoteSource

2012-01-24 Thread Sanne Grinovero
Hello, in the method: org.infinispan.distribution.DistributionManagerImpl.retrieveFromRemoteSource(Object, InvocationContext, boolean) we have: ListAddress targets = locate(key); // if any of the recipients has left the cluster since the command was issued, just don't wait for its