It's effectively a big problem. The last time I generated the doc for all maven-plugins, I needed 1Gb of memory !!
Arnaud > -----Message d'origine----- > De : Rafal Krzewski [mailto:[EMAIL PROTECTED] > Envoy� : mercredi 26 mai 2004 10:12 > � : Maven Developers List > Objet : Re: Maven 1.0 final work list > > > Brett Porter wrote: > > Hi, > > > > Apart from the documentation that I am working on at the > moment, is there > > anything else that needs to be completed for Maven 1.0? > > I'm sorry I haven't wrote about it earlier but I have been > really busy. > > I did some testing regarding > http://jira.codehaus.org/browse/MPMULTIPROJECT-19 bug, or more > specifically memory usage of the xdoc plugin. > > The results were not optimistic. The memory leak in xdoc is > outrageous. > Maven leaks about 50% of the memory it ever allocates. > > The test was done as follows. I took ledge-components project > http://objectledge.org/modules/ledge-components, cleaned out > the target, > then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of > markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, > under hprof profiler build into JDK. Settings were: allocation site > analysis, heap dump at exit, stacktrace depth 16. The run > took about 65 > minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was > 530MB virtual/240MB resident during the run, it surged some > additional > 300MB both during heap dump. The generated dump file is about 350MB, > 23MB when bzip2 compressed. I'm uploading the compressed dump to a > server right now so that people may look at it. I'll post an > URL on Jira. > > I'm going to post an xls file on Jira containg the statistics > of objects > allocated, and objects alive at the time of VM generation. > From what I'm > seeing none of the Jelly TagScript objects created gets ever garbage > collected. Wading through the heap dump I was able to > determine that all > of them are connected to the GC root through the main Thread, and a > ThreadLocal variable tagHolder declared in jelly's > TagScript.java, line > 115. Despite calls to tagHolder.set(null), the TagScript object stays > connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry > object which is a subclass of java.lang.ref.WeakReference. > ThreadLocal > implementation in JDK works like that. Now, AFAIK the GC is free to > harvest all objects that are not connected to a GC root through hard > reference, or a SoftReference. I wasn't able to trace all > references in > the object graph regarding TagScript objects (there are 21k+ > of them!) > so I am not sure if there are any other references that connect the > TagScripts to a GC root through a permanent refernce, like a static > field in some class. I cant tell you it's a dog's work to do that by > hand. If I only had a proper memory analysys tool! Alas all the good > ones are commercial... On the other hand GC is not *required* > to harvest > these objects. Maybe it just didn't bother to harvest because > it was not > pressed for memory? In my test it had still many megabytes > left. Maybe a > better test would be running it with too few memory and checking what > was left alive, despite the fact GC was desperate to find the space? > > At any rate, TagScript.tagHolder is suspicious, and if it's > even not the > culprit itself (could be if WeakRefernce implementation was > broken) it > is hiding the real problem (at least in the environment of my test). > Note that the variable is not static, which means all > TagScript object > instances get tied to the Thread. I don't know if the possibility of > using each instance concurrently from multiple threads is > useful at all, > especially that the instances have more state information, > not only the > tagHolder... Anyway, it's a wrong forum to discuss that. I also heard > that Jelly was abandoned by the original author so there is a good > chance that noone understands why this tagHolder thingy was > introduced > and what it does... > > Anyway. I think this memory leak this big is a shame. I call > upon those > who have experience in fighting memory problems in Java and > apropriate > tools at hand to step up and help fixing it before 1.0 hits > the streets. > > Rafal > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
