Re: C2 Memory Settings - Xms, Xmx, freememory, and heapsize
Hi Gerhard,
Thanks for the reply. I've done more work on this and posted two
messages on the cocoon-dev list, but your moderator hasn't let them
through yet! Quick summary is:
In the loop that frees an item from a store then calls the garbage
collector. Typically the item clears 5k (not m) of memory then the
garbage collector is called which takes 2.5 seconds. So, at this rate,
to clear 20m would take a few hours during which time the system appears
to be locked!
I've noticed when running Optimizeit that although garbage collection
can be quick, it sometimes takes a few seconds. If it has just been run,
and you force it to run again, then it seems to take longer. If it can't
find much to freeup, it seems to do a more time consuming search for
garbage.
Can the stores tell you how much they've freed (even approximately) so
you don't need to call the garbage collector then ask the jvm about
freememory so often?
Full details and evidence posted under dev list.
Peter
[EMAIL PROTECTED] wrote:
Hi Peter,
From: Peter Hargreaves [mailto:[EMAIL PROTECTED]]
Hi Gerhard,
In my earlier notes on the above topic, I mentioned my garbage collector
occasionally taking over the system thereby freezing it! I think it
Damn threads!
could be a bug connected with the Round Robin not managing to free
memory and not giving up trying to when it can't.
With the following settings:
Xms=0m
Xmx=140m
heapsize =1
freememory=2000
I find using Optimizit that when my actual heap get to just over 100m
and my heap in use gets over 80m, then usually memory gets freed as
expected. Sometimes at this point the lockup occurs. It usually happens
just after one or two attempts have only just managed to free more than
20m. If the store-janitor can't free memory (store empty) I would expect
it to let the memory in use continue to grow.
For that he checks if more then one store is registered.
Looking in my components.log and matching the output with the
StoreJanitorImpl.java source. The following seems to be happening:
Inside the method run() it seems to be getting stuck in the while loop
that repeatedly attempts to freeMemory() until the store is empty. It
doesn't seem to be able to empty the store!
while (this.memoryLow() this.getStoreList().size() 0) {
this.freeMemory();
}
In my log I find alternating groups of statements from this.memoryLow()
and this.freeMemory() and the message StoreList size=3 gets repeated.
Every attempt to free memory calls the garbage collector.
I couldn't understand the Round Robin thing so don't know if that is to
blame. There is other activity in the log which might be relevant, see
below.
Hmm I don't find the bug, yet. But I changed the logging messages at least.
Now it prints out memory is low true/false. Maybe that's more clearly-presented.
Could you do a new check-out of the Head version and redo your test?
I assume you just mean to try the latest CVS? I was trying to stick with
C2.0 because I've done so much testing with it for release of my
completed application. I'll see if I can organise myself to do this.
skip/
Gerhard
Children are hereditary;
if your parents didn't have any,
neither will you.
-
Please check that your question has not already been answered in the
FAQ before posting. http://xml.apache.org/cocoon/faqs.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
Please check that your question has not already been answered in the
FAQ before posting. http://xml.apache.org/cocoon/faqs.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: C2 Memory Settings - Xms, Xmx, freememory, and heapsize
Hi Gerhard, Many thanks for your time and the extra tips. So, following on the same thread, if you will bear with me? My 'real world' has the following 'special case': I want to deploy multiple independent cocoon applications in the same servlet container (Tomcat 4.0.1). Is this wise? I assume it means that my servlet container, and my cocoon applications will all be sharing the same jvm and the same heap. Method 1 - I could use sub-sitemaps. But, can this be done while using a separate WAR file to deploy each application? I couldn't find out how. Method 2 - Alternatively I could deploy cocoon within each application's WAR file. Although this is expensive in hard disk space, I don't mind the price in return for independence. However, it will result in multiple cocoon instances and I'm not so sure about multiple instances sharing the same java heap!! Should I be avoiding this? It might be incredibly expensive in RAM???!!! If I do create multiple instances of cocoon in the same jvm then what memory settings should I use? Xms - the initial size of the jvm heap. There seems to be no issue with this setting, I just set it to zero to avoid confusion and let the jvm increase the heap as it needs it. Xmx - the maximum size of the jvm heap. Again, I just set this to the maximum my system can make available to my servlet container and cocoon applications. But, I now have multiple instances of the store-janitor (don't I?), they are all monitoring the same jvm heap, but each with their own 'heapsize', 'freememory', 'cleanupthreadinterval' and 'priority' setting and running asynchronously. They will each set-off finalization and the garbage collector every interval, won't they? Although each cleanupthreadinterval could be increased, they will not be synchronised, so could all fire off close to each other leaving an unreasonable gap. This worries me, could it explain why my garbage collector occasionally takes over the system and crashes it (observed through Optimizeit)? Each store-janitor has its own store to look after, but is it possible to balance their demands without one taking outright priority over the others? store-janitor parameter 'heapsize' - Should all the heapsize params be set the same, or is there a case for setting them differently? I think the same, unless one application should be more concerned about it's use of memory than another. store-janitor parameter 'freememory' - Should all these be the same, or is there a case for setting them differently on each store-janitor. Again I think the same, as the results of setting them differently are harder to imagine! It seems to me that I should be avoiding multiple cocoon instances under the same jvm because the store-janitor is not really designed for it. Even if Xms is massive because of the over enthusiasm of garbage collection. Is this a reasonable conclusion? By the way, I tried just playing with the parameters and wasn't getting very far, but now I've become analytical as well, I'm making progress - in reducing my RAM requirement. Yes, Gerhard please make use of my notices for the documentation if you can but I suggest you use the following caveat: Listen to what I mean, not what I say. - pdh. Oh, I've just turned off logging and my application is transformed from Sluggish to Greased Lightning! - Wow!! Looking forward to your comments Gerhard. Comments from others also welcome. Regards, Peter. - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faqs.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: C2 Memory Settings - Xms, Xmx, freememory, and heapsize
Hi Peter, From: Peter Hargreaves [mailto:[EMAIL PROTECTED]] Hi Folks, I've completed my application and am investigating massive memory demands (100M - 300M). The following are some of my conclusions, which may be of use to others. I'm definitely not an expert so comments please from those that know better. Some of the following does seems to contradict other posting about memory configuration. Kool, mate. Keep up sending feedback! The java heap is the area of RAM reserved for use by the java virtual machine. At any time only part of the heap will be in use. The size of the part in use will vary over time as classes and instances are loaded and released. If the size in use approaches the size of the heap, then more memory will be allocated to the heap. If the amount in use falls to a small percentage of the heap size for a relatively long period the size of the heap will be reduced by the jvm, to free memory for the operating system. Xms and Xmx are set when the jvm is started. If using Tomcat-4.0.1 set the environment parameter CATALINA_OPTS=-Xmsvalue -Xmxvalue before starting Tomcat. Xms - sets initial java heap size - With cocoon there seems to be little point setting this as start-up times are dominated by other factors. Ok. Xmx - sets maximum java heap size - This is the maximum amount of memory available to Tomcat and Cocoon. If there is demand for more than is available, then a java.lang.outofmemory() exception will occur - to be avoided at all costs!!! I've found it best to set this to a size which leaves some RAM for the operating system. If there is virtual memory configured on the operating system (hard disk pretending to be RAM) then set the maximum jvm heap low enough to avoid using it. Yes I do the same. Store Janitor - In the cocoon.xconf file there are setting for the store-janitor. The store, I think, is a file cache run by cocoon which therefore places demands on the jvm heap. These setting control the size of the file cache to stop it using up all of the heap and thereby causing a java.lang.outofmemory() error. It's both. It's a memory store which hold the generated pages. When this cache is full or the JVM is running low on memory some objects are freed and swapped to the filesystem. But your're right the StoreJanitor is the component which controls this! store-janitor class=org.apache.cocoon.components.store.StoreJanitorImpl logger=root.store parameter name=freememory value=100/ parameter name=heapsize value=6000/ parameter name=cleanupthreadinterval value=10/ parameter name=threadpriority value=5/ /store-janitor parameter 'heapsize' - the size of jvm heap below which the file cache is allowed to grow. When actual heap size (not heap in use) exceeds this Exact! size then the store-janitor will take steps to reduce it, assuming the amount of actual free memory is below the parameter 'freememory' limit. The current alghorithm is like this: memory is low if: JVM total Memory consumption configured heap JVM free Memory configured free Memory Parameter 'heapsize' must be less than Xmx otherwise the store would be allowed to fill the heap. The difference between Xmx and 'heapsize' must be sufficient memory for cocoon during a 'cleanupthreadinterval'. Correct. parameter 'freememory' - is the difference between heap size and heap in use which the store-janitor must achieve (by giving up memory) if the actual heap size has exceeded the parameter 'heapsize'. This is the amount of free memory which must be available to cocoon during the 'cleanupthreadinterval' assuming the heapsize might no longer be able to increase. If 'freememory' is set too large, then the store-janitor will give up more of its cache than it needs to, or might not be able to free sufficient by emptying the file cache. parameter 'cleanupthreadinterval' - the period in seconds between store-janitor's attempts to monitor memory and adjust the store. Also triggers finalization and the garbage collector. Correct. My suggested settings for different ram sizes in Megabytes: (default)64128256384 ----------- Xms 0 0 0 0 Xmx 64100228356 heapsize60 80208236 freememory 1 20 2020 Note: The difference between Xmx and heapsize is needed to ensure sufficient free memory for cocoon during the first cleanupthreadinterval in which the actual heap size exceeds 'heapsize'. Similarly the parameter 'freememory' tells the store-janitor to attempt to clear sufficient memory for the next interval. So I've suggested the difference between Xms and heapsize be the same as the size of freememory. It might need to be bigger on a fast machine, or the interval reduced. Would the experts please comment
