Re: Virtual Memory very high
Are you seeing a real problem here, besides just being alarmed by the big numbers from top? Consumption of virtual memory by itself is basically harmless, as long as you're not running up against any of the OS limits (and, you're running a 64 bit JVM). This is just "top" telling you that you've mapped large files into the virtual memory space. It's not telling you that you don't have any RAM left... virtual memory is different from RAM. In my tests, generally MMapDirectory gives faster search performance than NIOFSDirectory... so unless there's an actual issue, I would recommend sticking with MMapDirectory. Mike McCandless http://blog.mikemccandless.com On Fri, Dec 9, 2011 at 11:54 PM, Rohit wrote: > Hi All, > > > > Don't know if this question is directly related to this forum, I am running > Solr in Tomcat on linux server. The moment I start tomcat the virtual memory > shown using TOP command goes to its max 31.1G and then remains there. > > > > Is this the right behaviour, why is the virtual memory usage so high. I have > 36GB of ram on the server. > > > > Tasks: 309 total, 1 running, 308 sleeping, 0 stopped, 0 zombie > > Cpu(s): 19.1%us, 0.2%sy, 0.0%ni, 79.3%id, 1.2%wa, 0.0%hi, 0.2%si, > 0.0%st > > Mem: 49555260k total, 36152224k used, 13403036k free, 121612k buffers > > Swap: 999416k total, 0k used, 999416k free, 5409052k cached > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 2741 mysql 20 0 6412m 5.8g 6380 S 182 12.3 108:07.45 mysqld > > 2814 root 20 0 31.1g 22g 9716 S 100 46.6 375:51.70 java > > 1765 root 20 0 12.2g 285m 9488 S 2 0.6 3:52.59 java > > 3591 root 20 0 19352 1576 1068 R 0 0.0 0:00.24 top > > 1 root 20 0 23684 1908 1276 S 0 0.0 0:06.21 init > > > > Regards, > > Rohit > > >
Re: Virtual Memory very high
Why do you care about virtual memory? It's after all, virtual. You can
allocate as much as you want.
For instance, MMapDirectory maps a load of virtual memory, but that
has little relation to how much physical memory is being used. Consider
looking at your app with something like jConsole and seeing how much
physical memory is being used before you worry about this issue.
Best
Erick
On Mon, Apr 2, 2012 at 4:56 AM, Suneel wrote:
> Hello Everyone,
>
> On window server.
>
> I am facing same problem during indexing my memory consumption going very
> high based on above discussion i checked in my Solrconfig.xml file and found
> that "directoryFactory" not configured yet. if i configuring
> directoryfactory then its will help me reduce the consumption of memory.
>
> i think below configuration used for linex server.
>
> class="${solr.directoryFactory:solr.NIOFSDirectoryFactory}"/>
>
> what will be best option for window server which solve my problem.
>
> Please suggest me.
>
>
>
>
>
>
> -
> Regards,
>
> Suneel Pandey
> Sr. Software Developer
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Virtual-Memory-very-high-tp3574817p3877097.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Virtual Memory very high
Hello Everyone, On window server. I am facing same problem during indexing my memory consumption going very high based on above discussion i checked in my Solrconfig.xml file and found that "directoryFactory" not configured yet. if i configuring directoryfactory then its will help me reduce the consumption of memory. i think below configuration used for linex server. what will be best option for window server which solve my problem. Please suggest me. - Regards, Suneel Pandey Sr. Software Developer -- View this message in context: http://lucene.472066.n3.nabble.com/Virtual-Memory-very-high-tp3574817p3877097.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Virtual Memory very high
On 12/13/2011 6:16 AM, Dmitry Kan wrote:
> If you allow me to chime in, is there a way to check for which
> DirectoryFactory is in use, if
> ${solr.directoryFactory:solr.StandardDirectoryFactory} has been configured?
I think you can get the currently used factory in a Luke response, if you hit
your Solr server with a Luke request,
eg http://localhost:8983/solr/admin/luke
>
> Dmitry
>
> 2011/12/12 Yury Kats
>
>> On 12/11/2011 4:57 AM, Rohit wrote:
>>> What are the difference in the different DirectoryFactory?
>>
>>
>> http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/MMapDirectory.html
>>
>> http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/NIOFSDirectory.html
>>
>
RE: Virtual Memory very high
Thanks Yurykats.
Regards,
Rohit
Mobile: +91-9901768202
About Me: http://about.me/rohitg
-Original Message-
From: Dmitry Kan [mailto:dmitry@gmail.com]
Sent: 13 December 2011 11:17
To: solr-user@lucene.apache.org
Subject: Re: Virtual Memory very high
If you allow me to chime in, is there a way to check for which
DirectoryFactory is in use, if
${solr.directoryFactory:solr.StandardDirectoryFactory} has been configured?
Dmitry
2011/12/12 Yury Kats
> On 12/11/2011 4:57 AM, Rohit wrote:
> > What are the difference in the different DirectoryFactory?
>
>
> http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/MMapDirectory.html
>
> http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/NIOFSDirectory.html
>
Re: Virtual Memory very high
If you allow me to chime in, is there a way to check for which
DirectoryFactory is in use, if
${solr.directoryFactory:solr.StandardDirectoryFactory} has been configured?
Dmitry
2011/12/12 Yury Kats
> On 12/11/2011 4:57 AM, Rohit wrote:
> > What are the difference in the different DirectoryFactory?
>
>
> http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/MMapDirectory.html
>
> http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/NIOFSDirectory.html
>
Re: Virtual Memory very high
On 12/11/2011 4:57 AM, Rohit wrote: > What are the difference in the different DirectoryFactory? http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/MMapDirectory.html http://lucene.apache.org/java/3_3_0/api/core/org/apache/lucene/store/NIOFSDirectory.html
RE: Virtual Memory very high
What are the difference in the different DirectoryFactory? Regards, Rohit Mobile: +91-9901768202 About Me: http://about.me/rohitg -Original Message- From: Yury Kats [mailto:yuryk...@yahoo.com] Sent: 10 December 2011 12:11 To: solr-user@lucene.apache.org Subject: Re: Virtual Memory very high On 12/9/2011 11:54 PM, Rohit wrote: > Hi All, > > > > Don't know if this question is directly related to this forum, I am running > Solr in Tomcat on linux server. The moment I start tomcat the virtual memory > shown using TOP command goes to its max 31.1G and then remains there. > > > > Is this the right behaviour, why is the virtual memory usage so high. I have > 36GB of ram on the server. To limit VIRT memory, change DirectoryFactory in the solrconfig.xml to use solr.NIOFSDirectoryFactory.
Re: Virtual Memory very high
On 12/9/2011 11:54 PM, Rohit wrote: > Hi All, > > > > Don't know if this question is directly related to this forum, I am running > Solr in Tomcat on linux server. The moment I start tomcat the virtual memory > shown using TOP command goes to its max 31.1G and then remains there. > > > > Is this the right behaviour, why is the virtual memory usage so high. I have > 36GB of ram on the server. To limit VIRT memory, change DirectoryFactory in the solrconfig.xml to use solr.NIOFSDirectoryFactory.
Virtual Memory very high
Hi All, Don't know if this question is directly related to this forum, I am running Solr in Tomcat on linux server. The moment I start tomcat the virtual memory shown using TOP command goes to its max 31.1G and then remains there. Is this the right behaviour, why is the virtual memory usage so high. I have 36GB of ram on the server. Tasks: 309 total, 1 running, 308 sleeping, 0 stopped, 0 zombie Cpu(s): 19.1%us, 0.2%sy, 0.0%ni, 79.3%id, 1.2%wa, 0.0%hi, 0.2%si, 0.0%st Mem: 49555260k total, 36152224k used, 13403036k free, 121612k buffers Swap: 999416k total,0k used, 999416k free, 5409052k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 2741 mysql 20 0 6412m 5.8g 6380 S 182 12.3 108:07.45 mysqld 2814 root 20 0 31.1g 22g 9716 S 100 46.6 375:51.70 java 1765 root 20 0 12.2g 285m 9488 S2 0.6 3:52.59 java 3591 root 20 0 19352 1576 1068 R0 0.0 0:00.24 top 1 root 20 0 23684 1908 1276 S0 0.0 0:06.21 init Regards, Rohit
