[
https://issues.apache.org/jira/browse/OAK-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6500:
---------------------------------
Attachment: OAK-6500-approach-a-v1.patch
[updated patch|^OAK-6500-approach-a-v1.patch] based on approach a. With this
the open file count comes closer to the start count
{noformat}
Open file count - At start 180
Open file count - Post creation of 100 nodes at /content/a is 297
Open file count - Post async run at /content/a is 300
Open file count - Post creation of 100 nodes at /content/b is 419
Open file count - Post async run at /content/b is 421
Open file count - Post creation of 100 nodes at /content/c is 539
Open file count - Post async run at /content/c is 422
Open file count - Post creation of 1 nodes at /content/d is 423
Open file count - Post async run at /content/d is 301
Open file count - Post creation of 1 nodes at /content/e is 302
Open file count - Post async run at /content/e is 181
Open file count - At end 181
{noformat}
Here the NRTIndex keep a memory of opened IndexReader and closes them upon
close of NRTIndex.
Downside here is that if for a sync lucene index if say 1000 commits happen
before AsyncIndexUpdate cycle is triggered then those 1000 reader instances
would be held in memory
Would now work on #B which would rely on ref counting
> NRTIndex leaks file handles due to unclosed IndexReader
> -------------------------------------------------------
>
> Key: OAK-6500
> URL: https://issues.apache.org/jira/browse/OAK-6500
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
> Affects Versions: 1.6.0
> Reporter: Chetan Mehrotra
> Assignee: Chetan Mehrotra
> Priority: Critical
> Fix For: 1.8, 1.7.5, 1.6.4
>
> Attachments: OAK-6500-approach-a-v1.patch, OAK-6500-v1.patch
>
>
> On some setups under stress it has been seen that NRTIndex leaks file handles
> over time.
> Checking with lsof indicates that more than 3 nrt folders per index are being
> used. However per design there can be max 3 and after system is not in use
> max 1 should be present.
> {noformat}
> $ lsof -p 9550 | grep '/nrt' | gawk 'match($0,
> /.*crx-quickstart\/repository\/index\/(.*?)\/\_.*$/, m) { print m[1]; }' |
> sort | uniq
> cqPageLucene-1501065263331/nrt1501065335930
> cqPageLucene-1501065263331/nrt1501065374667
> cqPageLucene-1501065263331/nrt1501065392492
> cqPageLucene-1501065263331/nrt1501065440955
> cqPageLucene-1501065263331/nrt1501065473286
> cqPageLucene-1501065263331/nrt1501065507345
> slingeventJob-1501065263330/nrt1501065356975
> slingeventJob-1501065263330/nrt1501065373229
> slingeventJob-1501065263330/nrt1501065394142
> slingeventJob-1501065263330/nrt1501065440953
> slingeventJob-1501065263330/nrt1501065473282
> slingeventJob-1501065263330/nrt1501065507342
> versionStoreIndex-1501065263332/nrt1501065335925
> versionStoreIndex-1501065263332/nrt1501065366781
> versionStoreIndex-1501065263332/nrt1501065392490
> versionStoreIndex-1501065263332/nrt1501065441232
> versionStoreIndex-1501065263332/nrt1501065473285
> {noformat}
> Further actually checking index folder indicates that those folder are
> actually deleted. So some where the file handle is still referring them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
