Hi Guys,

I think I've been leading myself up the garden path a bit here.  After
some more in-depth analysis I've noticed that this is directly related
to mem usage and garbage collection.  It would seem that the store
janitor and GC is kicking in at about the same time (which makes sense.)

The iowait is caused by a huge amount of memory being swapped out to
disk.

We're going to try with incremental GC turned on.  What is everyone else
doing as far as GC is concerned?  What is recognised as optimal store
janitor / gc config?  I suspect that this may be why so many users have
reported poor performance when using the persistent store.....

Of course, it's all anecdotal.

What I'd love to be able to do out of this research is produce a user's
guide to configuring these various components.  I think a document like
this should really form part of a release......

Corin
-----Original Message-----
From: Corin Moss
Sent: Thursday, 11 March 2004 3:55 p.m.
To: [EMAIL PROTECTED]
Subject: JCS Store - new issues


Hi Guys,
You may or may not have seen my recent posts about performance with JCS
store, but I've gotten a little further into diagnosing what looks like
a deadlock issue.
After about 30mins of load testing I'm seeing everything lock up (iowait
of ~ 100%) the error log is getting this message repeatedly:
ERROR   (2004-03-11) 15:24.32:587   [core.store.janitor] (Unknown-URI)
Unknown-thread/StoreJanitorImpl: Error in freeMemory()
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at java.util.ArrayList.RangeCheck(ArrayList.java:507)
        at java.util.ArrayList.get(ArrayList.java:324)
        at
org.apache.excalibur.store.impl.StoreJanitorImpl.freeMemory(StoreJanitor
Impl.java:383)
        at
org.apache.excalibur.store.impl.StoreJanitorImpl.run(StoreJanitorImpl.ja
va:202)
        at java.lang.Thread.run(Thread.java:534)
Obviously it's getting it every cycle of the Store Janitor.
I'm currently a little confused about it - it seems to me that the only
reason this error could be thrown is if the ArrayList which contains the
available stores is empty.  Is this the case?  If it is, then the next
question is:  what causes the JCS store to fall out of the store list?
Would this happen if JCS threw a fatal exception? I suspect it would,
but I'm not 100% certain.
I have a feeling I know the particular part of JCS that causes the
problem.  It seems to me that the R-W lock mechanism they use is not
quite perfect under load.
So - anyone have any more info on the questions above?
Thanks,
Corin
Corin Moss
Lead Developer
TVNZ Interactive
+64 9 916 7367
+64 21 403 054
[EMAIL PROTECTED]

================================================================
CAUTION: This e-mail and any attachment(s) contains information
that is intended to be read only by the named recipient(s). It
may contain information that is confidential, proprietary or the
subject of legal privilege. This information is not to be used by
any other person and/or organisation. If you are not the intended
recipient, please advise us immediately and delete this e-mail
from your system. Do not use any information contained in it.

================================================================
For more information on the Television New Zealand Group, visit
us online at http://www.tvnz.co.nz
================================================================

================================================================
CAUTION: This e-mail and any attachment(s) contains information
that is intended to be read only by the named recipient(s). It
may contain information that is confidential, proprietary or the
subject of legal privilege. This information is not to be used by
any other person and/or organisation. If you are not the intended
recipient, please advise us immediately and delete this e-mail
from your system. Do not use any information contained in it.

================================================================
For more information on the Television New Zealand Group, visit
us online at http://www.tvnz.co.nz
================================================================

Reply via email to