What about the following property? Is it still valid?
-Dsun.rmi.transport.tcp.connectionPool=true
Thanks,
Bryan
> -----Original Message-----
> From: Peter Firmstone [mailto:j...@zeus.net.au]
> Sent: Friday, January 13, 2012 8:41 AM
> To: Bryan Thompson
> Cc: dev@river.apache.org; u...@river.apache.org
> Subject: Re: [Fwd: Re: DGC threads issue]
>
> Brian,
>
> Do you have a Linux / Unix build environment?
>
> River isn't building yet on Window's, FreeBSD, or OpenBSD,
> known build environments that work are Ubuntu and Solaris.
> If not, that's ok, well be able to get the necessary jar's off Hudson.
>
> The DGC lease property values you need to set are:
>
> com.sun.jini.jeri.dgc.leaseValue=30000
> com.sun.jini.jeri.dgc.checkInterval=15000
>
> RMI (called JRMP) is superseded code now, River doesn't use
> that unless you configure your Exporter to be JrmpExporter,
> but you're better off sticking with BasicJeriExporter.
>
> Regards,
>
> Peter.
>
> Bryan Thompson wrote:
> > Peter,
> >
> > I would be happy to test out a fix on this. Can you point
> out the modified line and link me to any directions on how to
> rebuild the necessary jar(s)?
> >
> > Did you already file an issue for this? If not, I will
> file an issue and then update it when I test out this fix.
> >
> > Thanks,
> > Bryan
> >
> >
> >> -----Original Message-----
> >> From: Peter Firmstone [mailto:j...@zeus.net.au]
> >> Sent: Friday, January 13, 2012 3:21 AM
> >> To: dev@river.apache.org
> >> Cc: Bryan Thompson; u...@river.apache.org
> >> Subject: Re: [Fwd: Re: DGC threads issue]
> >>
> >> Ok, found the problem, when I fixed River-142, I introduced a new
> >> bug, after creating a new DGC thread, I didn't set the
> running flag
> >> to true, so each time, a new thread is created.
> >>
> >> Thankfully it's a very easy fix.
> >>
> >> Cheers,
> >>
> >> Peter.
> >>
> >> Peter Jones wrote:
> >>
> >>> Peter,
> >>>
> >>> That internal Executor interface permits neither
> >>>
> >> synchronous execution in the calling thread ("this method
> itself must
> >> not block") nor any small bound on the number of Runnables
> that can
> >> be executed concurrently ("execution of a given action must not be
> >> delayed indefinitely in order to complete execution of a different
> >> action passed to a different invocation of this method").
> In other
> >> words, concurrency-wise, it should be equivalent to "new
> >> Thread(runnable, name)", or what the result of
> >> j.u.c.Executors.newCachedThreadPool() would do.
> Otherwise, callers
> >> would not function correctly, because they expect to be
> able to pass
> >> Runnables that execute indefinitely, not just short-lived
> work tasks.
> >>
> >>> The real purpose of this internal Executor/ThreadPool was
> >>>
> >> to provide an internal alternative to "new Thread(...)" that takes
> >> care of certain things in common, like:
> >>
> >>> - thread reuse instead of creation when reasonable (idle timeout)
> >>> - but still with threads (re)named for the task being executed
> >>> - insulation of any thread creation permission requirement from
> >>> context calling execute (NewThreadAction)
> >>> - but instead requiring direct user of thread pool to be
> >>>
> >> trusted not
> >>
> >>> to abuse that insulation, with permission requirement for access
> >>> (ThreadPoolPermission and GetThreadPoolAction)
> >>> - common logging of uncaught exceptions
> >>>
> >>> -- Peter
> >>>
> >>>
> >>> On Jan 12, 2012, at 7:53 PM, Peter Firmstone wrote:
> >>>
> >>>
> >>>
> >>>> ThreadPool implements the com.sun.jini.thread.Executor interface.
> >>>>
> >>>> Because the interface states it should not block, I think
> >>>>
> >> the calling thread should execute the task when the thread pool
> >> becomes saturated, rather than continue to create new
> threads as per
> >> the current implementation. This will ensure that the task is
> >> completed, the calling thread, if it uses a sequence of tasks with
> >> dependencies will have to ensure that it submits the tasks
> in order.
> >> Since only the caller knows the order, it makes sense for
> this to be
> >> the callers responsibility. As a result the executor will
> honor the
> >> non blocking contract. For that reason we'll use a zero length
> >> queue, probably a SynchronousQueue.
> >>
> >>>> Regards,
> >>>>
> >>>> Peter.
> >>>>
> >>>> /**
> >>>> * Executor is an abstraction for a thread factory or
> >>>>
> >> thread pool for
> >>
> >>>> * executing actions asynchronously.
> >>>> *
> >>>> * @author Sun Microsystems, Inc.
> >>>> *
> >>>> */
> >>>> public interface Executor {
> >>>>
> >>>> /**
> >>>> * Executes the given Runnable action asynchronously in
> >>>>
> >> some thread.
> >>
> >>>> *
> >>>> * The implementation may create a new thread to execute
> >>>>
> >> the action,
> >>
> >>>> * or it may execute the action in an existing thread.
> >>>> *
> >>>> * The execution of a given action must not be delayed
> >>>>
> >> indefinitely
> >>
> >>>> * in order to complete execution of a different action
> >>>>
> >> passed to a
> >>
> >>>> * different invocation of this method. In other words, the
> >>>> * implementation must assume that there may be
> >>>>
> >> arbitrary dependencies
> >>
> >>>> * between actions passed to this method, so it needs to
> >>>>
> >> be careful
> >>
> >>>> * to avoid potential deadlock by delaying execution of
> >>>>
> >> one action
> >>
> >>>> * indefinitely until another completes.
> >>>> *
> >>>> * Also, this method itself must not block, because it
> >>>>
> >> may be invoked
> >>
> >>>> * by code that is serially processing data to produce
> >>>>
> >> multiple such
> >>
> >>>> * arbitrarily-dependent actions that need to be executed.
> >>>> *
> >>>> * @param runnable the Runnable action to execute
> >>>> *
> >>>> * @param name string to include in the name of the
> >>>>
> >> thread used
> >>
> >>>> * to execute the action
> >>>> */
> >>>> void execute(Runnable runnable, String name); }
> >>>>
> >>>> Peter Firmstone wrote:
> >>>>
> >>>>
> >>>>> Thanks Brian,
> >>>>>
> >>>>> Looking at our implementation code, DGC uses an Executor
> >>>>>
> >> called ThreadPool, it's javadoc states:
> >>
> >>>>> /**
> >>>>> * ThreadPool is a simple thread pool implementation of
> >>>>>
> >> the Executor
> >>
> >>>>> * interface.
> >>>>> *
> >>>>> * A new task is always given to an idle thread, if one is
> >>>>>
> >> available;
> >>
> >>>>> * otherwise, a new thread is always created. There is
> no minimum
> >>>>> * warm thread count, nor is there a maximum thread count
> >>>>>
> >> (tasks are
> >>
> >>>>> * never queued unless there are sufficient idle threads
> to execute
> >>>>> * them).
> >>>>> *
> >>>>> * New threads are created as daemon threads in the thread
> >>>>>
> >> group that
> >>
> >>>>> * was passed to the ThreadPool instance's constructor. Each
> >>>>> thread's
> >>>>> * name is the prefix NewThreadAction.NAME_PREFIX
> followed by the
> >>>>> name
> >>>>> * of the task it is currently executing, or "Idle" if it is
> >>>>> currently
> >>>>> * idle.
> >>>>>
> >>>>> ThreadPool predates Java 5, it looks like we can fix this
> >>>>>
> >> by using an Executor from Java 5, we can look at limiting
> the number
> >> of threads created based on available CPU's and a scaling
> factor and
> >> place the tasks in a BlockingQueue, so if the queue is filled, it
> >> blocks.
> >>
> >>>>> Can you report the issue as a Bug on Jira for me, I'll
> >>>>>
> >> fix this before the next release.
> >>
> >>>>> Regards,
> >>>>>
> >>>>> Peter.
> >>>>>
> >>>>> Peter Firmstone wrote:
> >>>>>
> >>>>>
> >>>>>> Hi Peter,
> >>>>>>
> >>>>>> I was wondering if you had any thoughts on this post
> >>>>>>
> >> from Bryan on River users?
> >>
> >>>>>> Hope you don't mind me asking ;)
> >>>>>>
> >>>>>> Best Regards,
> >>>>>>
> >>>>>> Peter Firmstone.
> >>>>>>
> >>>>>>
> >>>>>>
> >> -------------------------------------------------------------------
> >>
> >>>>>> -----
> >>>>>>
> >>>>>> Subject:
> >>>>>> Re: DGC threads issue
> >>>>>> From:
> >>>>>> Tom Hobbs <tvho...@googlemail.com>
> >>>>>> Date:
> >>>>>> Thu, 12 Jan 2012 20:45:01 +0000
> >>>>>> To:
> >>>>>> u...@river.apache.org, dev@river.apache.org
> >>>>>>
> >>>>>> To:
> >>>>>> u...@river.apache.org, dev@river.apache.org
> >>>>>>
> >>>>>>
> >>>>>> Hi Bryan,
> >>>>>>
> >>>>>> Sorry that no one got back to you about this. I'm
> afraid that I
> >>>>>> don't know the answer to your question, I've copied
> the dev list
> >>>>>> into this email in case someone who monitors that list
> (but not
> >>>>>> this one) has any ideas.
> >>>>>>
> >>>>>> Best regards,
> >>>>>>
> >>>>>> Tom
> >>>>>>
> >>>>>> On Thu, Jan 12, 2012 at 2:29 PM, Bryan Thompson
> >>>>>>
> >> <br...@systap.com> wrote:
> >>
> >>>>>>
> >>>>>>
> >>>>>>> Just to follow up on this thread myself. I modified
> >>>>>>>
> >> the pattern to return a "thick" future rather than a proxy for the
> >> future. This caused the RMI call to wait on the server until the
> >> future was done and then sent back the outcome. This
> "fixed" the DGC
> >> memory/thread leak by reducing the number of exported proxies
> >> drammatically.
> >>
> >>>>>>> In terms of best practices, is distributed DGC simply
> >>>>>>>
> >> not useful for exported objects with short life spans?
> Can it only
> >> be used with proxies for relatively long lived services?
> >>
> >>>>>>> Thanks,
> >>>>>>> Bryan
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Bryan Thompson
> >>>>>>>> Sent: Tuesday, January 03, 2012 12:06 PM
> >>>>>>>> To: u...@river.apache.org
> >>>>>>>> Subject: DGC threads issue
> >>>>>>>>
> >>>>>>>> Hello,
> >>>>>>>>
> >>>>>>>> Background:
> >>>>>>>>
> >>>>>>>> I am seeing what would appear to be one DGC thread
> >>>>>>>>
> >> allocated per
> >>
> >>>>>>>> exported object. This is using River 2.2 and Sun JDK
> >>>>>>>>
> >> 1.6.0_17.
> >>
> >>>>>>>> Relevant configuration parameters are below.
> >>>>>>>>
> >>>>>>>> I am observing problems with the DGC threads not being
> >>>>>>>>
> >> retired on
> >>
> >>>>>>>> a timely basis. The exported objects are proxies
> for Futures
> >>>>>>>> which are being executed on the service. The code
> pattern is
> >>>>>>>> such that the proxied Future goes out of lexical scope quite
> >>>>>>>> quickly. E.g., rmiCallReturningProxyForFuture().get().
> >>>>>>>>
> >>>>>>>> Under a modest load, a large number of such Futures
> >>>>>>>>
> >> are exported
> >>
> >>>>>>>> which results in a large number of long lived DGC
> >>>>>>>>
> >> threads. This
> >>
> >>>>>>>> turns into a problem for the JVM due to the stack
> >>>>>>>>
> >> allocation per
> >>
> >>>>>>>> thread. Presumably this is not good for other
> reasons as well
> >>>>>>>> (e.g., scheduling).
> >>>>>>>>
> >>>>>>>> I have tried to override the leaseValue and checkInterval
> >>>>>>>> defaults per the configuration options below. I
> >>>>>>>>
> >> suspect that the
> >>
> >>>>>>>> lease interval is somehow not being obeyed, which is
> >>>>>>>>
> >> presumably a
> >>
> >>>>>>>> problem on my end. However, I can verify that the
> >>>>>>>>
> >> configuration
> >>
> >>>>>>>> values are in fact showing up in
> >>>>>>>> System.getProperties() for at least some of the JVMs
> involved
> >>>>>>>> (the one which drives the workload and the one that I am
> >>>>>>>> monitoring with the large number of DGC lease threads).
> >>>>>>>>
> >>>>>>>> Some questions:
> >>>>>>>>
> >>>>>>>> Is this one-thread-per-exported proxy the expected
> >>>>>>>>
> >> behavior when
> >>
> >>>>>>>> DGC is requested for the exported object?
> >>>>>>>>
> >>>>>>>> The DGC lease checker threads appear to expire ~14 -
> >>>>>>>>
> >> 15 minutes
> >>
> >>>>>>>> after I terminate the process which was originating the RMI
> >>>>>>>> requests. This is close the sum of the default
> >>>>>>>>
> >> leaseValue (10m)
> >>
> >>>>>>>> and checkInterval (5m) parameters, but maybe there is
> >>>>>>>>
> >> some other
> >>
> >>>>>>>> timeout which is controlling this? If this is the sum
> >>>>>>>>
> >> of those
> >>
> >>>>>>>> parameters, why would the DGC lease threads live until
> >>>>>>>>
> >> the sum of
> >>
> >>>>>>>> those values? I thought that the lease would expire
> after the
> >>>>>>>> leaseValue (10m default).
> >>>>>>>>
> >>>>>>>> Can the issue I am observing be caused by a low heap
> >>>>>>>>
> >> pressure on
> >>
> >>>>>>>> the JVM to which the RMI proxies were exported? If it
> >>>>>>>>
> >> fails to
> >>
> >>>>>>>> GC those proxies, even though they are reachable, could that
> >>>>>>>> cause DGC to continue to retain those proxies on the
> JVM which
> >>>>>>>> exported them?
> >>>>>>>>
> >>>>>>>> Is there any way to configure DGC to use a thread pool
> >>>>>>>>
> >> or to have
> >>
> >>>>>>>> the leases managed by a single thread?
> >>>>>>>>
> >>>>>>>> Is it possible that there is an interaction with the
> >>>>>>>>
> >> useNIO option?
> >>
> >>>>>>>> Relevant options that I am using include:
> >>>>>>>>
> >>>>>>>> -Dcom.sun.jini.jeri.tcp.useNIO=true
> >>>>>>>> -Djava.rmi.dgc.leaseValue=30000
> >>>>>>>> -Dsun.rmi.dgc.checkInterval=15000
> >>>>>>>> -Dsun.rmi.transport.tcp.connectionPool=true
> >>>>>>>>
> >>>>>>>> Thanks in advance,
> >>>>>>>> Bryan
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
>
>
