I don't thought that the IdleRemover itself has a problem.
I thought the managed connections must somehow incorporate a thread.
The number of threads which got lost was always a multiple of
the listed LocalManagedConnectionFactorys (which always was a
multiple of 5).
Anyway, I attach my UsageLogger-MBean, which is a first result of
my experiments towards a health-monitor.
The getTreadCount()-method should do. It was a (paranoid) attempt
to count threads like they show up in the threaddump.
I tried and got the same result with a more simple implementation,
but I'll leave it the paranoid way (counting each ThreadGroup separatly).
Regards
Michael Bartmann
David Jencks wrote:
> This is rather strange since there is supposed to be one IdleRemover thread
> systemwide. I'll take a look. Can you give me any more hints on how to
> count threads?
>
> thanks
> david jencks
>
> On 2002.09.23 12:45:04 -0400 Michael Bartmann wrote:
>
>>I increased the log-level to debug, and found that exactly at the time
>>the thread counts increase (and become more inconsistent), there was the
>>message:
>>
>>2002-09-23 18:13:10,015 DEBUG
>>[org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover
>>notifying pools, interval: 450000
>>2002-09-23 18:13:10,015 DEBUG
>>[org.jboss.resource.adapter.jdbc.local.LocalManagedConnectionFactory]
>>Using properties: {user=sa, password=}
>>(last line repeats 10 times)
>>
>>This might have something to do with our own JMSProvider, which uses
>>JDBC as its persistence layer.
>>Cause it has its own XA implementation it uses a db-connection not
>>enlisted under under the tx-mgr.
>>This non-transacted db-connection comes out of a NoTxDatasource which I
>>created via the XSLSubDeployer.
>>
>>I suspect that I did something wrong (too much copy/paste?) when
>>modifying the XSL-code from LocalTX.
>>(This is the modification/addition I have commited to cvs some weeks
>>ago).
>>There is no thread which our JMSProvider directly creates at that moment
>>in time, so I think I triggered a bug
>>in the jvm (non-started threads, see previous posting) or a questionable
>>use of such threads by the pool
>>or by the jdbc-driver.
>>I experience this with oracle-thin _and_ hypersonic which provides
>>evidence against a jdbc issue.
>>
>>This might be a job for an expert in jca and pooling issues (David?).
>>
>>Thank you,
>>Michael Bartmann
>>
>>
>>
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Jboss-development mailing list
>>[EMAIL PROTECTED]
>>https://lists.sourceforge.net/lists/listinfo/jboss-development
>>
>>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Jboss-development mailing list
> [EMAIL PROTECTED]
> https://lists.sourceforge.net/lists/listinfo/jboss-development
>
package de.j4production.jboss300.health;
/**
* <p>Title: </p>
* <p>Description: </p>
* <p>Copyright: Copyright (c) 2002</p>
* <p>Company: </p>
* @author unbekannt
* @version 1.0
*/
public class UsageData
{
private long timeOfMessure;
private long memoryConsumption;
private int threadCount;
private int jvmThreadCount;
public long getMemoryConsumption()
{
return memoryConsumption;
}
public int getJvmThreadCount()
{
return jvmThreadCount;
}
public int getThreadCount()
{
return threadCount;
}
public long getTimeOfMessure()
{
return timeOfMessure;
}
public void setTimeOfMessure(long timeOfMessure)
{
this.timeOfMessure = timeOfMessure;
}
public void setThreadCount(int threadCount)
{
this.threadCount = threadCount;
}
public void setJvmThreadCount(int jvmThreadCount)
{
this.jvmThreadCount = jvmThreadCount;
}
public void setMemoryConsumption(long memoryConsumption)
{
this.memoryConsumption = memoryConsumption;
}
}
package de.j4production.jboss300.health;
import java.util.*;
import java.text.*;
import org.jboss.system.*;
import org.jboss.logging.*;
/**
* @author Michael Bartmann
* @version 1.0
*/
public class UsageLogger extends ServiceMBeanSupport implements UsageLoggerMBean
{
private Logger log = Logger.getLogger(UsageLogger.class);
protected final static NumberFormat i2fmt = new DecimalFormat("00");
protected final static NumberFormat i3fmt = new DecimalFormat("000");
protected final static NumberFormat i4fmt = new DecimalFormat("0000");
protected final static NumberFormat cntfmt = new DecimalFormat("#,##0");
private Thread worker;
private boolean shouldRun;
private long sleepMillis = 15000;
private List dataPoints = new ArrayList();
private class Collector implements Runnable
{
public void run()
{
try
{
long lastTime = System.currentTimeMillis();
long nextTime = lastTime;
while (shouldRun)
{
long now = System.currentTimeMillis();
long diff = now - nextTime;
if (diff > 100)
{
log.warn(this+": too late ("+diff+" ms)");
nextTime += diff;
}
else if (diff < -100)
{
log.warn(this+": too early ("+(-diff)+" ms)");
}
if (log.isTraceEnabled()) log.trace(this+": collecting data");
collectData();
lastTime = nextTime;
nextTime = lastTime + sleepMillis;
long toSleep = nextTime - System.currentTimeMillis() + 1;
while ((toSleep>0)&&(shouldRun))
{
try
{
if (log.isTraceEnabled()) log.trace(this+": sleeping for
"+toSleep+" ms");
Thread.sleep(toSleep);
}
catch (InterruptedException ex)
{
log.info(this+": was interrupted.");
}
nextTime = lastTime + sleepMillis;
toSleep = nextTime - System.currentTimeMillis();
}
}
}
catch (Throwable t)
{
log.fatal(this+": unexpected problems",t);
}
}
}
private void collectData()
{
UsageData data = new UsageData();
data.setTimeOfMessure(System.currentTimeMillis());
data.setMemoryConsumption(getUsedMemory());
data.setThreadCount(getThreadCount());
data.setJvmThreadCount(getJVMThreadCount());
dataPoints.add(data);
}
public int getMessureCount()
{
return dataPoints.size();
}
protected void startService() throws Exception
{
if ((worker!=null)&&(!worker.isAlive()))
{
worker = null;
}
if (worker==null)
{
worker = new Thread(new Collector());
worker.setDaemon(true);
worker.setName("Usage Logger");
worker.start();
}
shouldRun = true;
}
protected void stopService() throws Exception
{
shouldRun = false;
if ((worker!=null)&&(worker.isAlive()))
{
worker.interrupt();
worker.join(3000);
}
}
protected void destroyService() throws Exception
{
shouldRun = false;
if ((worker!=null)&&(worker.isAlive()))
{
worker.interrupt();
worker.join(3000);
}
}
public long getUsedMemory()
{
return Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
}
public int getJVMThreadCount()
{
return rootThreadGroup().activeCount();
}
public int getThreadCount()
{
return getThreadCount(rootThreadGroup());
}
protected int getThreadCount(ThreadGroup g)
{
int sum = 0;
int maxt = g.activeCount() + 100;
Thread[] ta = new Thread[maxt];
g.enumerate(ta,false);
for (int i=0; i<maxt; i++)
{
if (ta[i]!=null)
{
sum += 1;
}
}
int maxg = g.activeGroupCount() + 100;
ThreadGroup[] tg = new ThreadGroup[maxg];
g.enumerate(tg,false);
for (int i=0; i<maxg; i++)
{
if (tg[i]!=null)
{
sum += getThreadCount(tg[i]);
}
}
return sum;
}
protected ThreadGroup rootThreadGroup()
{
ThreadGroup g = Thread.currentThread().getThreadGroup();
while (g.getParent()!=null)
{
g = g.getParent();
}
return g;
}
public long getSleepMillis()
{
return sleepMillis;
}
public void setSleepMillis(long sleepMillis)
{
log.info(this+" adjusting sleepMillis to "+sleepMillis+" ms");
this.sleepMillis = sleepMillis;
if (worker!=null)
{
worker.interrupt();
}
}
public String dumpUsageAsHTML(int cnt)
{
StringBuffer buf = new StringBuffer();
buf.append("<HTML>\n");
buf.append(" <TABLE BORDER=\"1\" CELLPADDING=\"2\" CELLSPACING=\"0\">\n");
buf.append(" <TR>\n");
buf.append(" <TH>data#</TH>\n");
buf.append(" <TH>time</TH>\n");
buf.append(" <TH>bytes used</TH>\n");
buf.append(" <TH>threads</TH>\n");
buf.append(" <TH>threads(jvm)</TH>\n");
buf.append(" </TR>\n");
int i = 0;
if (cnt<dataPoints.size())
{
i = dataPoints.size()-cnt;
}
while (i<dataPoints.size())
{
UsageData data = (UsageData) dataPoints.get(i++);
buf.append(" <TR>\n");
buf.append(" <TD>"+i+"</TD>\n");
buf.append("
<TD>"+timestamp2YMDHMS(data.getTimeOfMessure())+"</TD>\n");
buf.append(" <TD
ALIGN=\"RIGHT\">"+cntfmt.format(data.getMemoryConsumption())+"</TD>\n");
buf.append(" <TD
ALIGN=\"RIGHT\">"+cntfmt.format(data.getThreadCount())+"</TD>\n");
buf.append(" <TD
ALIGN=\"RIGHT\">"+cntfmt.format(data.getJvmThreadCount())+"</TD>\n");
buf.append(" </TR>\n");
}
buf.append(" </TABLE>\n");
buf.append("</HTML>\n");
return buf.toString();
}
protected static String timestamp2YMDHMS(long timestamp)
{
StringBuffer h = new StringBuffer();
Calendar gcal = GregorianCalendar.getInstance();
gcal.setTime(new Date(timestamp));
h.append(i4fmt.format(gcal.get(GregorianCalendar.YEAR)));
h.append('-');
h.append(i2fmt.format(gcal.get(GregorianCalendar.MONTH)+1));
h.append('-');
h.append(i2fmt.format(gcal.get(GregorianCalendar.DAY_OF_MONTH)));
h.append(' ');
h.append(i2fmt.format(gcal.get(GregorianCalendar.HOUR_OF_DAY)));
h.append(':');
h.append(i2fmt.format(gcal.get(GregorianCalendar.MINUTE)));
h.append(':');
h.append(i2fmt.format(gcal.get(GregorianCalendar.SECOND)));
h.append(',');
h.append(i3fmt.format(gcal.get(GregorianCalendar.MILLISECOND)));
return h.toString();
}
}
package de.j4production.jboss300.health;
import javax.management.ObjectName;
import org.w3c.dom.Element;
import org.w3c.dom.Document;
import org.jboss.util.jmx.ObjectNameFactory;
import org.jboss.system.Service;
import org.jboss.system.ServiceMBean;
/**
* <p>Title: </p>
* <p>Description: </p>
* <p>Copyright: Copyright (c) 2002</p>
* <p>Company: </p>
* @author unbekannt
* @version 1.0
*/
public interface UsageLoggerMBean extends Service, ServiceMBean
{
public long getUsedMemory();
public int getJVMThreadCount();
public int getThreadCount();
public int getMessureCount();
public long getSleepMillis();
public void setSleepMillis(long sleepMillis);
public String dumpUsageAsHTML(int cnt);
}
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE server>
<server>
<mbean code="de.j4production.jboss300.health.UsageLogger"
name="fs4p:service=UsageLogger">
<attribute name="SleepMillis">15000</attribute>
</mbean>
</server>
