On 9/18/2012 9:29 PM, Lance Norskog wrote:
There is a known JVM garbage collection bug that causes this. It has to do with
reclaiming Weak references, I think in WeakHashMap. Concurrent garbage
collection collides with this bug and the result is that old field cache data
is retained after closing the index. The bug is more common with more
processors doing GC simultaneously.
The symptom is that when you run a monitor, the memory usage rises to a peak,
drops to a floor, rises again in the classic sawtooth pattern. When the GC bug
happens, the ceiling becomes the floor, and the sawtooth goes from the new
floor to a new ceiling. The two sizes are the same. So, 2G to 5G, over and
over, suddenly it is 5G to 8G, over and over.
The bug is fixed in recent Java 7 releases. I'm sorry, but I cannot find the
bug number.
I think I ran into this when I was looking at memory usage on my SolrJ
indexing program. Under Java6, memory usage in jconsole (remotely via
JMX) was fairly constant long-term (aside from the unavoidable
sawtooth). When I ran it under Java 7u3, it would continually grow,
slowly ... but if I measured it with jstat on the Linux commandline
rather than remotely via jconsole under windows, memory usage was
consistent over time, just like under java6 with the remote jconsole.
After looking at heap dumps and scratching my head a lot, I finally
concluded that I did not have a memory leak, there was a problem with
remote JMX monitoring in java7. Glad to hear I was not imagining it,
and that it's fixed now.
Thanks,
Shawn
