Re: Virtual memory usage of elasticsearch
Thanks Jorg. I will look into that. On Wednesday, 20 August 2014 13:20:38 UTC+5:30, Jörg Prante wrote: If you get OOM, you should take a look at RSS (resident set size) of the process. The VIRT (virtual memory) can span GB or TB, it does not matter at all. Jörg On Wed, Aug 20, 2014 at 9:36 AM, Gokul nath gokula...@gmail.com javascript: wrote: Hi, when i see top command virtual memory usage for ElasticSearch is high. I have allocated min 1G and max 2G heap size for it. How can i control the virtual memory usage? I am getting OOM frequently on ElasticSearch log. PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 3721 elastics 200 87.4g 2.1g 119m S 66.214.3 29:45.90 java Thanks Gokul -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4fbb3426-f8d3-4cee-a6d6-d0d8003fd8b9%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/4fbb3426-f8d3-4cee-a6d6-d0d8003fd8b9%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/341063ae-4a61-4ea9-a308-3d3005c8e019%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Virtual memory usage of elasticsearch
Hi, when i see top command virtual memory usage for ElasticSearch is high. I have allocated min 1G and max 2G heap size for it. How can i control the virtual memory usage? I am getting OOM frequently on ElasticSearch log. PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 3721 elastics 200 87.4g 2.1g 119m S 66.214.3 29:45.90 java Thanks Gokul -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4fbb3426-f8d3-4cee-a6d6-d0d8003fd8b9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
how to use cluster.routing.allocation.allow_rebalance ?
Hi,
I am trying to ensure that re-allocation of shards are prevented whenever
one of the nodes in the cluster goes down. Based on the documentation I read
online, I use the following settings on the cluster
persistent: {
cluster.routing.allocation.enable: none,
cluster.routing.allocation.disable_allocation:true
}
However, this prevents new indices from being allocated as well. This is not
OK for me and so my requirement is to be able to do both the following
things -
1. Be able to create indices dynamically.
2. Prevent re-allocation of shards when a node in the cluster goes down.
Based on the documentation, I understood that setting
cluster.routing.allocation.allow_rebalance to indices_all_active will do
both the things. But this didn't happen. Hence, my understanding is clearly
wrong. So what does it mean to set allow_rebalance to indices_all_active ?
When going over the mailing list, I also found the concept of dynamic
settings. I assume this means that there are some values that I can set
when the cluster is live and some settings that I can only set in the
configuration file, which will be set on restarting elastic search. Is
allow_rebalance one such setting?
Please correct me if I am on the wrong path altogether.
I see that I have raised multiple questions in one thread. However, all of
them are a related use case to me. Can you please advice me on these.
Thanks,
--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/how-to-use-cluster-routing-allocation-allow-rebalance-tp4061785.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1407924508318-4061785.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.
Re: Marvel Indices taking lot of space ? Can we specify automatic delete of marvel indice ?
Deepak, Can you try the command with a hyphen at the end - /usr/lib/python2.6/site-packages/curator/curator.py -d 3 -p .marvel- For some reason, curator doesn't match the marvel indices when using .marvel as the prefix. Regards, On Thursday, May 15, 2014 1:14:14 AM UTC+8, Deepak Subhramanian wrote: Hi Ivan/Boaz, I am getting some warning when I tried to delete the marvel index using curator. github]# /usr/lib/python2.6/site-packages/curator/curator.py -d 3 -p .marvel 2014-05-14T18:07:44.691 INFOmain:333 Job starting... 2014-05-14T18:07:44.692 INFO _new_conn:180 Starting new HTTP connection (1): localhost 2014-05-14T18:07:44.694 INFO log_request_success:49 GET http://localhost:9200/ [status:200 request:0.003s] 2014-05-14T18:07:44.695 INFOmain:359 Deleting indices older than 3 days... 2014-05-14T18:07:44.698 INFO log_request_success:49 GET http://localhost:9200/.marvel*/_settings?expand_wildcards=closed[status:200 request:0.002s] 2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.07 2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.08 2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.09 2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.10 2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.11 2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.12 2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.13 2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not find a valid timestamp from the index: .marvel-2014.05.14 2014-05-14T18:07:44.723 INFO index_loop:309 DELETE index operations completed. 2014-05-14T18:07:44.723 INFOmain:379 Done in 0:00:00.037963. [root@node05 github]# ls -ltr /var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel* /var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.10: total 8 drwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 10 01:00 _state drwxr-xr-x. 5 elasticsearch elasticsearch 4096 May 10 01:00 0 /var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.11: Any thoughts? On Tue, May 13, 2014 at 12:26 PM, deepakas [via ElasticSearch Users] [hidden email] http://user/SendEmail.jtp?type=nodenode=4055895i=0 wrote: Hi Ivan/Boaz, Thanks for the responses. I will checkout Curator. I already got the latest version of marvel last week using this command. bin/plugin -i elasticsearch/marvel/latest Thanks, Deepak -- If you reply to this email, your message will be added to the discussion below: http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729p4055784.html To unsubscribe from Marvel Indices taking lot of space ? Can we specify automatic delete of marvel indice ?, click here. NAMLhttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- Deepak Subhramanian -- View this message in context: Re: Marvel Indices taking lot of space ? Can we specify automatic delete of marvel indice ?http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729p4055895.html Sent from the ElasticSearch Users mailing list archivehttp://elasticsearch-users.115913.n3.nabble.com/at Nabble.com. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/36baa5ec-c467-47df-b597-0aee612dd888%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
