[
https://issues.apache.org/jira/browse/LUCENE-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677281#action_12677281
]
rviper edited comment on LUCENE-1186 at 2/27/09 3:53 AM:
-----------------------------------------------------------------
i'm using quartz schedules to trigger indexing tasks
since the analyzer class is using a thread local -> should i reuse an analyzer
(e.g singleton pattern) in this case (quartz job reusing threads (=tread pool)
-> thread local data never gets freed)...)?
Mem dump fragment (after out of memory):
Class name Shallow Heap Retained HeapPercentage
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e8070
QuartzScheduler_Worker-3 Thread 120 55.914.144 21,65%
|- java.lang.ThreadLocal$ThreadLocalMap @ 0xdf32a20 24 55.913.760 21,65%
| '- java.lang.ThreadLocal$ThreadLocalMap$Entry[16384] @ 0x163a61b8
65.552 55.913.736 21,65%
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe090ca8 32 9.608 0,00%
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe0f6b88 ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe5d3df8 ....
org.apache.lucene.analysis.StopAnalyzer$SavedStreams @ 0xe090cc8 24
9.560 0,00%
org.apache.lucene.analysis.LowerCaseTokenizer @ 0xe090ce0 32
8.520 0,00%
org.apache.lucene.analysis.CharArraySet @ 0xe090e30 24 968
0,00%
org.apache.lucene.analysis.StopFilter @ 0xe090e18 24 24
0,00%
org.apache.lucene.analysis.StopAnalyzer @ 0xe0911f8 24 24
0,00%
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe629d80 ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xeb53510 ....
......
......
quartz overview (eclipse memory analysis):
Class name Shallow Heap | Retained Heap | Percentage
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e8070
QuartzScheduler_Worker-3 Thread| 120 | 55.914.144 | 21,65%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e7a50
QuartzScheduler_Worker-7 Thread| 120 | 30.684.056 | 11,88%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e7710
QuartzScheduler_Worker-9 Thread| 120 | 19.464.024 | 7,54%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e7ee8
QuartzScheduler_Worker-4 Thread| 120 | 14.813.640 | 5,74%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e78c8
QuartzScheduler_Worker-8 Thread| 120 | 11.154.576 | 4,32%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e81f8
QuartzScheduler_Worker-2 Thread| 120 | 8.403.544 | 3,25%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e8380
QuartzScheduler_Worker-1 Thread| 120 | 8.334.552 | 3,23%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e7d60
QuartzScheduler_Worker-5 Thread| 120 | 8.314.904 | 3,22%
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e8508
QuartzScheduler_Worker-0 Thread| 120 | 8.267.376 | 3,20%
was (Author: rviper):
i'm using quartz schedules to trigger indexing tasks
since the analyzer class is using a thread local -> should i reuse an analyzer
(e.g singleton pattern) in this case (quartz job reusing threads (=tread pool)
-> thread local data never gets freed)...)?
Mem dump (part):
Class name Shallow Heap Retained HeapPercentage
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e8070
QuartzScheduler_Worker-3 Thread 120 55.914.144 21,65%
|- java.lang.ThreadLocal$ThreadLocalMap @ 0xdf32a20 24 55.913.760 21,65%
| '- java.lang.ThreadLocal$ThreadLocalMap$Entry[16384] @ 0x163a61b8
65.552 55.913.736 21,65%
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe090ca8 32 9.608 0,00%
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe0f6b88 ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe5d3df8 ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe629d80 ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xeb53510 ....
......
......
> [PATCH] Clear ThreadLocal instances in close()
> ----------------------------------------------
>
> Key: LUCENE-1186
> URL: https://issues.apache.org/jira/browse/LUCENE-1186
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 2.4.1, 2.9
>
> Attachments: LUCENE-1186-SegmentReader.patch, LUCENE-1186.patch
>
>
> As already found out in LUCENE-436, there seems to be a garbage collection
> problem with ThreadLocals at certain constellations, resulting in an
> OutOfMemoryError.
> The resolution there was to remove the reference to the ThreadLocal value
> when calling the close() method of the affected classes (see FieldsReader and
> TermInfosReader).
> For Java < 5.0, this can effectively be done by calling
> threadLocal.set(null); for Java >= 5.0, we would call threadLocal.remove()
> Analogously, this should be done in *any* class which creates ThreadLocal
> values
> Right now, two classes of the core API make use of ThreadLocals, but do not
> properly remove their references to the ThreadLocal value
> 1. org.apache.lucene.index.SegmentReader
> 2. org.apache.lucene.analysis.Analyzer
> For SegmentReader, I have attached a simple patch.
> For Analyzer, there currently is no patch because Analyzer does not provide a
> close() method (future to-do?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
