[jira] [Commented] (LUCENE-5254) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks
[ https://issues.apache.org/jira/browse/LUCENE-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791126#comment-13791126 ] ASF subversion and git services commented on LUCENE-5254: - Commit 1530845 from [~rcmuir] in branch 'dev/branches/lucene_solr_4_5' [ https://svn.apache.org/r1530845 ] LUCENE-4998, LUCENE-5242, LUCENE-5254, LUCENE-5262, LUCENE-5263, LUCENE-5264: svn merge -c 1522723 -c 1525896 -c 1529136 -c 1529141 -c 1530063 -c 1530416 -c 1530657 SegmentCoreReader's owner reference back to the first SegmentReader causes leaks -- Key: LUCENE-5254 URL: https://issues.apache.org/jira/browse/LUCENE-5254 Project: Lucene - Core Issue Type: Bug Affects Versions: 4.6, 5.0 Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.6, 5.0 Attachments: LUCENE-5254.patch Spinoff from LUCENE-5248, where Shai discovered this ... SegmentCoreReaders has a SegmentReader owner member, that points to the first SegmentReader that was opened. When that SR is reopened to SR2, e.g. because new deletes or NDV updates happened, the same SCR is shared. But, even if you close SR1, any thing it points to cannot be GCd because SCR is pointing to it. I think the big things are liveDocs and the NDV update maps; Shai is going to fix the latter in LUCENE-5248, so this issue should fix liveDocs. The simplest fix is to make liveDocs not final and null it out in doClose ... but that's sort of fragile (what if we add other members in the future and forget to null them on close?). I think it'd be better to eliminate the owner reference; it's only used so we can evict FieldCache entry once the core is closed. Maybe we can just store the coreCacheKey instead? -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5254) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks
[ https://issues.apache.org/jira/browse/LUCENE-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786085#comment-13786085 ] ASF subversion and git services commented on LUCENE-5254: - Commit 1529135 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1529135 ] LUCENE-5254: don't hold ref to original SR from SCR, to avoid bounded leak of things like live docs bitset SegmentCoreReader's owner reference back to the first SegmentReader causes leaks -- Key: LUCENE-5254 URL: https://issues.apache.org/jira/browse/LUCENE-5254 Project: Lucene - Core Issue Type: Bug Affects Versions: 5.0, 4.6 Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5254.patch Spinoff from LUCENE-5248, where Shai discovered this ... SegmentCoreReaders has a SegmentReader owner member, that points to the first SegmentReader that was opened. When that SR is reopened to SR2, e.g. because new deletes or NDV updates happened, the same SCR is shared. But, even if you close SR1, any thing it points to cannot be GCd because SCR is pointing to it. I think the big things are liveDocs and the NDV update maps; Shai is going to fix the latter in LUCENE-5248, so this issue should fix liveDocs. The simplest fix is to make liveDocs not final and null it out in doClose ... but that's sort of fragile (what if we add other members in the future and forget to null them on close?). I think it'd be better to eliminate the owner reference; it's only used so we can evict FieldCache entry once the core is closed. Maybe we can just store the coreCacheKey instead? -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5254) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks
[ https://issues.apache.org/jira/browse/LUCENE-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786087#comment-13786087 ] ASF subversion and git services commented on LUCENE-5254: - Commit 1529136 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1529136 ] LUCENE-5254: don't hold ref to original SR from SCR, to avoid bounded leak of things like live docs bitset SegmentCoreReader's owner reference back to the first SegmentReader causes leaks -- Key: LUCENE-5254 URL: https://issues.apache.org/jira/browse/LUCENE-5254 Project: Lucene - Core Issue Type: Bug Affects Versions: 5.0, 4.6 Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, 4.6 Attachments: LUCENE-5254.patch Spinoff from LUCENE-5248, where Shai discovered this ... SegmentCoreReaders has a SegmentReader owner member, that points to the first SegmentReader that was opened. When that SR is reopened to SR2, e.g. because new deletes or NDV updates happened, the same SCR is shared. But, even if you close SR1, any thing it points to cannot be GCd because SCR is pointing to it. I think the big things are liveDocs and the NDV update maps; Shai is going to fix the latter in LUCENE-5248, so this issue should fix liveDocs. The simplest fix is to make liveDocs not final and null it out in doClose ... but that's sort of fragile (what if we add other members in the future and forget to null them on close?). I think it'd be better to eliminate the owner reference; it's only used so we can evict FieldCache entry once the core is closed. Maybe we can just store the coreCacheKey instead? -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5254) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks
[ https://issues.apache.org/jira/browse/LUCENE-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786101#comment-13786101 ] ASF subversion and git services commented on LUCENE-5254: - Commit 1529139 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1529139 ] LUCENE-5254: just pass 'this' to the CoreClosedListeners SegmentCoreReader's owner reference back to the first SegmentReader causes leaks -- Key: LUCENE-5254 URL: https://issues.apache.org/jira/browse/LUCENE-5254 Project: Lucene - Core Issue Type: Bug Affects Versions: 5.0, 4.6 Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, 4.6 Attachments: LUCENE-5254.patch Spinoff from LUCENE-5248, where Shai discovered this ... SegmentCoreReaders has a SegmentReader owner member, that points to the first SegmentReader that was opened. When that SR is reopened to SR2, e.g. because new deletes or NDV updates happened, the same SCR is shared. But, even if you close SR1, any thing it points to cannot be GCd because SCR is pointing to it. I think the big things are liveDocs and the NDV update maps; Shai is going to fix the latter in LUCENE-5248, so this issue should fix liveDocs. The simplest fix is to make liveDocs not final and null it out in doClose ... but that's sort of fragile (what if we add other members in the future and forget to null them on close?). I think it'd be better to eliminate the owner reference; it's only used so we can evict FieldCache entry once the core is closed. Maybe we can just store the coreCacheKey instead? -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5254) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks
[ https://issues.apache.org/jira/browse/LUCENE-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786104#comment-13786104 ] ASF subversion and git services commented on LUCENE-5254: - Commit 1529141 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1529141 ] LUCENE-5254: just pass 'this' to the CoreClosedListeners SegmentCoreReader's owner reference back to the first SegmentReader causes leaks -- Key: LUCENE-5254 URL: https://issues.apache.org/jira/browse/LUCENE-5254 Project: Lucene - Core Issue Type: Bug Affects Versions: 5.0, 4.6 Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, 4.6 Attachments: LUCENE-5254.patch Spinoff from LUCENE-5248, where Shai discovered this ... SegmentCoreReaders has a SegmentReader owner member, that points to the first SegmentReader that was opened. When that SR is reopened to SR2, e.g. because new deletes or NDV updates happened, the same SCR is shared. But, even if you close SR1, any thing it points to cannot be GCd because SCR is pointing to it. I think the big things are liveDocs and the NDV update maps; Shai is going to fix the latter in LUCENE-5248, so this issue should fix liveDocs. The simplest fix is to make liveDocs not final and null it out in doClose ... but that's sort of fragile (what if we add other members in the future and forget to null them on close?). I think it'd be better to eliminate the owner reference; it's only used so we can evict FieldCache entry once the core is closed. Maybe we can just store the coreCacheKey instead? -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5254) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks
[ https://issues.apache.org/jira/browse/LUCENE-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13784070#comment-13784070 ] Simon Willnauer commented on LUCENE-5254: - +1 to use the core key only. I checked what ES does with it and we always pull the core key here anyways. SegmentCoreReader's owner reference back to the first SegmentReader causes leaks -- Key: LUCENE-5254 URL: https://issues.apache.org/jira/browse/LUCENE-5254 Project: Lucene - Core Issue Type: Bug Affects Versions: 5.0, 4.6 Reporter: Michael McCandless Spinoff from LUCENE-5248, where Shai discovered this ... SegmentCoreReaders has a SegmentReader owner member, that points to the first SegmentReader that was opened. When that SR is reopened to SR2, e.g. because new deletes or NDV updates happened, the same SCR is shared. But, even if you close SR1, any thing it points to cannot be GCd because SCR is pointing to it. I think the big things are liveDocs and the NDV update maps; Shai is going to fix the latter in LUCENE-5248, so this issue should fix liveDocs. The simplest fix is to make liveDocs not final and null it out in doClose ... but that's sort of fragile (what if we add other members in the future and forget to null them on close?). I think it'd be better to eliminate the owner reference; it's only used so we can evict FieldCache entry once the core is closed. Maybe we can just store the coreCacheKey instead? -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5254) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks
[ https://issues.apache.org/jira/browse/LUCENE-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13784445#comment-13784445 ] Uwe Schindler commented on LUCENE-5254: --- +1 To nuke this reference and only use the core cache key! Please dont make things in SegmentReaders unfinal! This is bad design! It took some time until Shai and I understood why RAMUsageEstimator showed so much RAM. But this was a real good use case for the Estimator. Because it showd the bug :-) SegmentCoreReader's owner reference back to the first SegmentReader causes leaks -- Key: LUCENE-5254 URL: https://issues.apache.org/jira/browse/LUCENE-5254 Project: Lucene - Core Issue Type: Bug Affects Versions: 5.0, 4.6 Reporter: Michael McCandless Spinoff from LUCENE-5248, where Shai discovered this ... SegmentCoreReaders has a SegmentReader owner member, that points to the first SegmentReader that was opened. When that SR is reopened to SR2, e.g. because new deletes or NDV updates happened, the same SCR is shared. But, even if you close SR1, any thing it points to cannot be GCd because SCR is pointing to it. I think the big things are liveDocs and the NDV update maps; Shai is going to fix the latter in LUCENE-5248, so this issue should fix liveDocs. The simplest fix is to make liveDocs not final and null it out in doClose ... but that's sort of fragile (what if we add other members in the future and forget to null them on close?). I think it'd be better to eliminate the owner reference; it's only used so we can evict FieldCache entry once the core is closed. Maybe we can just store the coreCacheKey instead? -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
