You can't hold the ThreadLocal value in a WeakReference, because
there is no hard reference between enumeration calls (so it would be
cleared out from under you while enumerating).
All of this occurs because you have some objects (readers/segments
etc.) that are shared across all threads, but these contain objects
that are 'thread/search state' specific. These latter objects are
essentially "cached" for performance (so you don't need to seek and
read, sequential buffer access, etc.)
A sometimes better solution is to have the state returned to the
caller, and require the caller to pass/use the state later - then you
don't need thread locals.
You can accomplish a similar solution by returning a "SessionKey"
object, and have the caller pass this later. You can then have a
WeakHashMap of SessionKey,SearchState that the code can use. When
the SessionKey is destroyed (no longer referenced), the state map can
be cleaned up automatically.
On Sep 10, 2008, at 11:30 PM, Noble Paul നോബിള്
नोब्ळ् wrote:
When I look at the reference tree That is the feeling I get. if you
held a WeakReference it would get released .
|- base of org.apache.lucene.index.CompoundFileReader$CSIndexInput
|- input of org.apache.lucene.index.SegmentTermEnum
|- value of java.lang.ThreadLocal$ThreadLocalMap
$Entry
On Wed, Sep 10, 2008 at 8:39 PM, Chris Lu <[EMAIL PROTECTED]> wrote:
Does this make any difference?
If I intentionally close the searcher and reader failed to release
the
memory, I can not rely on some magic of JVM to release it.
--
Chris Lu
-------------------------
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com/index.php?
title=Create_Lucene_Database_Search_in_3_minutes
DBSight customer, a shopping comparison site, (anonymous per
request) got
2.6 Million Euro funding!
On Wed, Sep 10, 2008 at 4:03 AM, Noble Paul
നോബിള് नोब्ळ्
<[EMAIL PROTECTED]> wrote:
Why do you need to keep a strong reference?
Why not a WeakReference ?
--Noble
On Wed, Sep 10, 2008 at 12:27 AM, Chris Lu <[EMAIL PROTECTED]>
wrote:
The problem should be similar to what's talked about on this
discussion.
http://lucene.markmail.org/message/keosgz2c2yjc7qre?q=ThreadLocal
There is a memory leak for Lucene search from Lucene-1195.(svn
r659602,
May23,2008)
This patch brings in a ThreadLocal cache to TermInfosReader.
It's usually recommended to keep the reader open, and reuse it when
possible. In a common J2EE application, the http requests are
usually
handled by different threads. But since the cache is
ThreadLocal, the
cache
are not really usable by other threads. What's worse, the cache
can not
be
cleared by another thread!
This leak is not so obvious usually. But my case is using
RAMDirectory,
having several hundred megabytes. So one un-released resource is
obvious
to
me.
Here is the reference tree:
org.apache.lucene.store.RAMDirectory
|- directory of org.apache.lucene.store.RAMFile
|- file of org.apache.lucene.store.RAMInputStream
|- base of
org.apache.lucene.index.CompoundFileReader$CSIndexInput
|- input of org.apache.lucene.index.SegmentTermEnum
|- value of java.lang.ThreadLocal$ThreadLocalMap
$Entry
After I switched back to svn revision 659601, right before this
patch is
checked in, the memory leak is gone.
Although my case is RAMDirectory, I believe this will affect
disk based
index also.
--
Chris Lu
-------------------------
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com/index.php?
title=Create_Lucene_Database_Search_in_3_minutes
DBSight customer, a shopping comparison site, (anonymous per
request)
got
2.6 Million Euro funding!
--
--Noble Paul
--------------------------------------------------------------------
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
--Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
