Hi, @Galder, no I didn't. I will take a look.
I was not aware of the ManagedBlocker interface but it fits on our problem. The best thing would be if we execute the local and remote operations on the F/J pool so I could wrap all the blocking invocations in ManagedBlocker without having to check if the operation is local or not... but that is subject for another topic :) Thanks for pointing it. Pedro On 11/13/2014 01:08 PM, William Burns wrote: > On Thu, Nov 13, 2014 at 7:59 AM, Dan Berindei <dan.berin...@gmail.com> wrote: >> Radim, I also knew the 1.7 ForkJoinPool isn't really optimized for blocking >> tasks, but the ManagedBlocker interface mentioned in [3] seems to be >> intended just for that. > > I was actually writing this same thing about ManagedBlocker instance > in the fork join thread for operations that will block (acquiring > locks - RPC etc.) The cool thing is that it then creates another > thread to try to keep CPU utilization higher and then after the > blocking is done it removes the next idle thread. Also it is in Java > 7 as well, your comment seemed a bit ambiguous as to whether or not > you were saying it wasn't. > >> >> Re: commonPool(), we can (and should) still create our own ForkJoinPool >> instead of using the global one. > > I am not familiar with this issue, but if true sounds like an issue > with ForkJoinPool implementation that we should log to the JDK. > > I haven't been able to dig into the code underneath but it sounded > like from the Javadoc that the common pool would eventually shut down > the worker threads after inactivity which should free the additional > references. "Using the common pool normally reduces resource usage > (its threads are slowly reclaimed during periods of non-use, and > reinstated upon subsequent use)." > >> >> Cheers >> Dan >> >> >> On Thu, Nov 13, 2014 at 9:57 AM, Radim Vansa <rva...@redhat.com> wrote: >>> >>> F/J tasks should not acquire any locks (or, generally, block) during >>> their execution. At least according to JavaDocs. Are we ready for that? >>> >>> Btw., I really don't like the fact that the commonPool() cannot be >>> properly shutdown. This leads to threadlocal variables leaking when the >>> component using F/J pool is undeployed (the classloader cannot be GCed >>> and you end up with OOME in PermGen space). >>> >>> Radim >>> >>> On 11/13/2014 08:28 AM, Galder Zamarreño wrote: >>>> @Pedro, did you consider using a ForkJoinPool instead? >>>> >>>> Traditional JDK pools are known to be very hard to configure and get it >>>> “right”. Fork join pools are being used as default thread pools in other >>>> libraries, vastly reducing configuration. >>>> >>>> Jessitron has published some interesting blog posts on the advantages of >>>> traditional ExecutorService vs Fork/Join pools and viceversa. See [1] and >>>> [3]. She also did a talk on it, see [4]. >>>> >>>> Cheers, >>>> >>>> p.s. I’ve not studied your use case in depth to decide whether F/J would >>>> suite better, but it’s certainly worth a look now that we’re on Java 7. >>>> >>>> [1] >>>> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ForkJoinPool.html >>>> [2] http://blog.jessitron.com/2014/01/choosing-executorservice.html >>>> [3] >>>> http://blog.jessitron.com/2014/02/scala-global-executioncontext-makes.html >>>> [4] https://www.youtube.com/watch?v=yhguOt863nw >>>> >>>> On 07 Nov 2014, at 09:31, Radim Vansa <rva...@redhat.com> wrote: >>>> >>>>> Btw., have you ever considered checks if a thread returns to pool >>>>> reasonably often? Some of the other datagrids use this, though there's >>>>> not much how to react upon that beyond printing out stack traces (but >>>>> you can at least report to management that some node seems to be >>>>> broken). >>>>> >>>>> Radim >>>>> >>>>> On 11/07/2014 08:35 AM, Bela Ban wrote: >>>>>> That's exactly what I suggested. No config gives you a shared global >>>>>> thread pool for all caches. >>>>>> >>>>>> Those caches which need a separate pool can do that via configuration >>>>>> (and of course also programmatically) >>>>>> >>>>>> On 06/11/14 20:31, Tristan Tarrant wrote: >>>>>>> My opinion is that we should aim for less configuration, i.e. >>>>>>> threadpools should mostly have sensible defaults and be shared by >>>>>>> default unless there are extremely good reasons for not doing so. >>>>>>> >>>>>>> Tristan >>>>>>> >>>>>>> On 06/11/14 19:40, Radim Vansa wrote: >>>>>>>> I second the opinion that any threadpools should be shared by >>>>>>>> default. >>>>>>>> There are users who have hundreds or thousands of caches and having >>>>>>>> separate threadpool for each of them could easily drain resources. >>>>>>>> And >>>>>>>> sharing resources is the purpose of threadpools, right? >>>>>>>> >>>>>>>> Radim >>>>>>>> >>>>>>>> On 11/06/2014 04:37 PM, Bela Ban wrote: >>>>>>>>> #1 I would by default have 1 thread pool shared by all caches >>>>>>>>> #2 This global thread pool should be configurable, perhaps in the >>>>>>>>> <global> section ? >>>>>>>>> #3 Each cache by default uses the gobal thread pool >>>>>>>>> #4 A cache can define its own thread pool, then it would use this >>>>>>>>> one >>>>>>>>> and not the global thread pool >>>>>>>>> >>>>>>>>> I think this gives you a mixture between ease of use and >>>>>>>>> flexibility in >>>>>>>>> configuring pool per cache if needed >>>>>>>>> >>>>>>>>> On 06/11/14 16:23, Pedro Ruivo wrote: >>>>>>>>>> On 11/06/2014 03:01 PM, Bela Ban wrote: >>>>>>>>>>> On 06/11/14 15:36, Pedro Ruivo wrote: >>>>>>>>>>>> * added a single thread remote executor service. This will >>>>>>>>>>>> handle the >>>>>>>>>>>> FIFO deliver commands. Previously, they were handled by JGroups >>>>>>>>>>>> incoming >>>>>>>>>>>> threads and with a new executor service, each cache can process >>>>>>>>>>>> their >>>>>>>>>>>> own FIFO commands concurrently. >>>>>>>>>>> +1000. This allows multiple updates from the same sender but to >>>>>>>>>>> different caches to be executed in parallel, and will speed thing >>>>>>>>>>> up. >>>>>>>>>>> >>>>>>>>>>> Do you intend to share a thread pool between the invocations >>>>>>>>>>> handlers of >>>>>>>>>>> the various caches, or do they each have their own thread pool ? >>>>>>>>>>> Or is >>>>>>>>>>> this configurable ? >>>>>>>>>>> >>>>>>>>>> That is question that cross my mind and I don't have any idea what >>>>>>>>>> would >>>>>>>>>> be the best. So, for now, I will leave the thread pool shared >>>>>>>>>> between >>>>>>>>>> the handlers. >>>>>>>>>> >>>>>>>>>> Never thought to make it configurable, but maybe that is the best >>>>>>>>>> option. And maybe, it should be possible to have different >>>>>>>>>> max-thread >>>>>>>>>> size per cache. For example: >>>>>>>>>> >>>>>>>>>> * all caches using this remote executor will share the same >>>>>>>>>> instance >>>>>>>>>> <remote-executor name="shared" shared="true" max-threads=4.../> >>>>>>>>>> >>>>>>>>>> * all caches using this remote executor will create their own >>>>>>>>>> thread >>>>>>>>>> pool with max-threads equals to 1 >>>>>>>>>> <remote-executor name="low-throughput-cache" shared="false" >>>>>>>>>> max-threads=1 .../> >>>>>>>>>> >>>>>>>>>> * all caches using this remote executor will create their own >>>>>>>>>> thread >>>>>>>>>> pool with max-threads equals to 1000 >>>>>>>>>> <remote executor name="high-throughput-cache" shared="false" >>>>>>>>>> max-thread=1000 .../> >>>>>>>>>> >>>>>>>>>> is this what you have in mind? comments? >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Pedro >>>>>>>>>> _______________________________________________ >>>>>>>>>> infinispan-dev mailing list >>>>>>>>>> infinispan-dev@lists.jboss.org >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>> >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> infinispan-dev@lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>> >>>>> >>>>> -- >>>>> Radim Vansa <rva...@redhat.com> >>>>> JBoss DataGrid QA >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev@lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> -- >>>> Galder Zamarreño >>>> gal...@redhat.com >>>> twitter.com/galderz >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev@lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> -- >>> Radim Vansa <rva...@redhat.com> >>> JBoss DataGrid QA >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev