Re: [infinispan-dev] Remoting package refactor

2014-11-14 Thread Pedro Ruivo
Hi, @Galder, no I didn't. I will take a look. I was not aware of the ManagedBlocker interface but it fits on our problem. The best thing would be if we execute the local and remote operations on the F/J pool so I could wrap all the blocking invocations in ManagedBlocker without having to check

Re: [infinispan-dev] Remoting package refactor

2014-11-13 Thread William Burns
On Thu, Nov 13, 2014 at 7:59 AM, Dan Berindei wrote: > Radim, I also knew the 1.7 ForkJoinPool isn't really optimized for blocking > tasks, but the ManagedBlocker interface mentioned in [3] seems to be > intended just for that. I was actually writing this same thing about ManagedBlocker instance

Re: [infinispan-dev] Remoting package refactor

2014-11-13 Thread Dan Berindei
Radim, I also knew the 1.7 ForkJoinPool isn't really optimized for blocking tasks, but the ManagedBlocker interface mentioned in [3] seems to be intended just for that. Re: commonPool(), we can (and should) still create our own ForkJoinPool instead of using the global one. Cheers Dan On Thu, No

Re: [infinispan-dev] Remoting package refactor

2014-11-12 Thread Radim Vansa
F/J tasks should not acquire any locks (or, generally, block) during their execution. At least according to JavaDocs. Are we ready for that? Btw., I really don't like the fact that the commonPool() cannot be properly shutdown. This leads to threadlocal variables leaking when the component using

Re: [infinispan-dev] Remoting package refactor

2014-11-12 Thread Galder Zamarreño
@Pedro, did you consider using a ForkJoinPool instead? Traditional JDK pools are known to be very hard to configure and get it “right”. Fork join pools are being used as default thread pools in other libraries, vastly reducing configuration. Jessitron has published some interesting blog posts

Re: [infinispan-dev] Remoting package refactor

2014-11-08 Thread Dan Berindei
I don't think we'll ever get to the point where we don't need *any* thread pools in Infinispan :) OTOH I also want to reduce the number of thread pools and thread pool configurations, so I'd rather not add per-cache thread pools until we see a clear need for it. In particular, I don't think we ne

Re: [infinispan-dev] Remoting package refactor

2014-11-07 Thread Sanne Grinovero
I think our priority should be to get rid of the need for threadpools - not their configuration options. If there is a real need for threadpools, then you have to provide full configuration options as you simply don't know how it's going to be used nor on what kind of hardware it's going to be run

Re: [infinispan-dev] Remoting package refactor

2014-11-07 Thread Bela Ban
Hi Radim, no I haven't. However, you can replace the thread pools used by JGroups and use custom pools. I like another idea better: inject Byteman code at runtime that keeps track of this, and *other useful stats as well*. It would be very useful to support if we could ship a package to a cus

Re: [infinispan-dev] Remoting package refactor

2014-11-07 Thread Radim Vansa
Btw., have you ever considered checks if a thread returns to pool reasonably often? Some of the other datagrids use this, though there's not much how to react upon that beyond printing out stack traces (but you can at least report to management that some node seems to be broken). Radim On 11/0

Re: [infinispan-dev] Remoting package refactor

2014-11-06 Thread Bela Ban
That's exactly what I suggested. No config gives you a shared global thread pool for all caches. Those caches which need a separate pool can do that via configuration (and of course also programmatically) On 06/11/14 20:31, Tristan Tarrant wrote: > My opinion is that we should aim for less conf

Re: [infinispan-dev] Remoting package refactor

2014-11-06 Thread Tristan Tarrant
My opinion is that we should aim for less configuration, i.e. threadpools should mostly have sensible defaults and be shared by default unless there are extremely good reasons for not doing so. Tristan On 06/11/14 19:40, Radim Vansa wrote: > I second the opinion that any threadpools should be s

Re: [infinispan-dev] Remoting package refactor

2014-11-06 Thread Gustavo Fernandes
> I second the opinion that any threadpools should be shared by default. > There are users who have hundreds or thousands of caches and having > separate threadpool for each of them could easily drain resources. And > sharing resources is the purpose of threadpools, right? Provided that no interde

Re: [infinispan-dev] Remoting package refactor

2014-11-06 Thread Radim Vansa
I second the opinion that any threadpools should be shared by default. There are users who have hundreds or thousands of caches and having separate threadpool for each of them could easily drain resources. And sharing resources is the purpose of threadpools, right? Radim On 11/06/2014 04:37 PM

Re: [infinispan-dev] Remoting package refactor

2014-11-06 Thread Bela Ban
#1 I would by default have 1 thread pool shared by all caches #2 This global thread pool should be configurable, perhaps in the section ? #3 Each cache by default uses the gobal thread pool #4 A cache can define its own thread pool, then it would use this one and not the global thread pool I th

Re: [infinispan-dev] Remoting package refactor

2014-11-06 Thread Pedro Ruivo
On 11/06/2014 03:01 PM, Bela Ban wrote: > > > On 06/11/14 15:36, Pedro Ruivo wrote: >> >> * added a single thread remote executor service. This will handle the >> FIFO deliver commands. Previously, they were handled by JGroups incoming >> threads and with a new executor service, each cache can pr

Re: [infinispan-dev] Remoting package refactor

2014-11-06 Thread Bela Ban
On 06/11/14 15:36, Pedro Ruivo wrote: > Hello, > > As many of you should know, I'm refactoring the remoting package. I was > trying and testing what can be done and it's time for an update. > Finally, I got a working version (needs some cleanups) that can be found > here [1]. > > The goal is to r

[infinispan-dev] Remoting package refactor

2014-11-06 Thread Pedro Ruivo
Hello, As many of you should know, I'm refactoring the remoting package. I was trying and testing what can be done and it's time for an update. Finally, I got a working version (needs some cleanups) that can be found here [1]. The goal is to reduce the complexity from CommandAwareRpcDispatcher