[jira] [Commented] (LUCENE-5928) WildcardQuery may has memory leak
[ https://issues.apache.org/jira/browse/LUCENE-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14140089#comment-14140089 ] Uwe Schindler commented on LUCENE-5928: --- Hi, as I mentioned before, SHR is shared memory - used by multiple processes. In the case of mmap, it is shared memory used by the kernel (FS cache) and at the same time by the process (mmapped files currently in fs cache), so you cannot count it 2 times. In fact, if you use mmap, the "real" resident memory by a process is RES - SHR. The remaining SHR is just resident (because allocated) because the kernel has actually done this allocation and frees it on request (there must be something that allocates RAM for the disk cache...). It is a little bit confusing, but correct :-) There is nothing wrong with Solr. > WildcardQuery may has memory leak > - > > Key: LUCENE-5928 > URL: https://issues.apache.org/jira/browse/LUCENE-5928 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 4.9 > Environment: SSD 1.5T, RAM 256G, records 15*1*1 >Reporter: Littlestar >Assignee: Uwe Schindler > Attachments: idxb1-top-sorted-mem.png, top_java.jpg > > > data 800G, records 15*1*1. > one search thread. > content:??? > content:* > content:*1 > content:*2 > content:*3 > jvm heap=96G, but the jvm memusage over 130g? > run more wildcard, use memory more > Does luence search/index use a lot of DirectMemory or Native Memory? > I use -XX:MaxDirectMemorySize=4g, it does nothing better. > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5928) WildcardQuery may has memory leak
[ https://issues.apache.org/jira/browse/LUCENE-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139919#comment-14139919 ] Shawn Heisey commented on LUCENE-5928: -- I have seen this exact memory reporting issue on my Solr installs. Java appears to misreport memory usage. See how your SHR value is 37GB? If you subtract that from your RES value, I think that's a lot closer to how much memory it's actually using. For what follows, reference the screenshot that I have attached. !idxb1-top-sorted-mem.png! Solr on this machine has a max heap of 6144M. The machine has 64GB of RAM. Let's add up some numbers, each of which I would ordinarily consider to be absolute truth: 16GB: RES size of the Solr process. 9.8GB: The amount of memory listed as free. 44GB: The amount of memory used by the OS disk cache. 1.1GB: next largest java process. 0.3GB: next largest java process. These numbers add up to more than 70GB ... but the machine only has 64GB total. > WildcardQuery may has memory leak > - > > Key: LUCENE-5928 > URL: https://issues.apache.org/jira/browse/LUCENE-5928 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 4.9 > Environment: SSD 1.5T, RAM 256G, records 15*1*1 >Reporter: Littlestar >Assignee: Uwe Schindler > Attachments: idxb1-top-sorted-mem.png, top_java.jpg > > > data 800G, records 15*1*1. > one search thread. > content:??? > content:* > content:*1 > content:*2 > content:*3 > jvm heap=96G, but the jvm memusage over 130g? > run more wildcard, use memory more > Does luence search/index use a lot of DirectMemory or Native Memory? > I use -XX:MaxDirectMemorySize=4g, it does nothing better. > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5928) WildcardQuery may has memory leak
[ https://issues.apache.org/jira/browse/LUCENE-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126913#comment-14126913 ] Uwe Schindler commented on LUCENE-5928: --- You may also look at: http://stackoverflow.com/questions/561245/virtual-memory-usage-from-java-under-linux-too-much-memory-used > WildcardQuery may has memory leak > - > > Key: LUCENE-5928 > URL: https://issues.apache.org/jira/browse/LUCENE-5928 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 4.9 > Environment: SSD 1.5T, RAM 256G 10*1 >Reporter: Littlestar >Assignee: Uwe Schindler > > data 800G, records 15*1*1. > one search thread. > content:??? > content:* > content:*1 > content:*2 > content:*3 > jvm heap=96G, but the jvm memusage over 130g? > run more wildcard, use memory more > Does luence search/index use a lot of DirectMemory or Native Memory? > I use -XX:MaxDirectMemorySize=4g, it does nothing better. > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5928) WildcardQuery may has memory leak
[ https://issues.apache.org/jira/browse/LUCENE-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126800#comment-14126800 ] Littlestar commented on LUCENE-5928: when I changed to NIO, It works OK. Does MMAP use a lot of NativeMemory? > WildcardQuery may has memory leak > - > > Key: LUCENE-5928 > URL: https://issues.apache.org/jira/browse/LUCENE-5928 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 4.9 > Environment: SSD 1.5T, RAM 256G 10*1 >Reporter: Littlestar > > data 800G, records 15*1*1. > one search thread. > content:??? > content:* > content:*1 > content:*2 > content:*3 > jvm heap=96G, but the jvm memusage over 130g? > run more wildcard, use memory more > Does luence search/index use a lot of DirectMemory or Native Memory? > I use -XX:MaxDirectMemorySize=4g, it does nothing better. > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
