Re: Virtual memory usage of elasticsearch

[Gokul nath]

Thanks Jorg. I will look into that. 

On Wednesday, 20 August 2014 13:20:38 UTC+5:30, Jörg Prante wrote:
>
> If you get OOM, you should take a look at RSS (resident set size) of the 
> process. The VIRT (virtual memory) can span GB or TB, it does not matter at 
> all.
>
> Jörg
>
>
> On Wed, Aug 20, 2014 at 9:36 AM, Gokul nath  > wrote:
>
>> Hi,
>>
>> when i see top command virtual memory usage for ElasticSearch is high. I 
>> have allocated min 1G and max 2G heap size for it. How can i control the 
>> virtual memory usage? I am getting OOM frequently on ElasticSearch log.
>>
>>   PID USER  PR  NI  VIRT  RES  SHR   S   %CPU %MEMTIME+  COMMAND
>>  3721 elastics  200  87.4g   2.1g 119m  S   66.214.3 
>> 29:45.90 java
>>
>> Thanks
>> Gokul
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/4fbb3426-f8d3-4cee-a6d6-d0d8003fd8b9%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/elasticsearch/4fbb3426-f8d3-4cee-a6d6-d0d8003fd8b9%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/341063ae-4a61-4ea9-a308-3d3005c8e019%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Virtual memory usage of elasticsearch

[Gokul nath]

Hi,

when i see top command virtual memory usage for ElasticSearch is high. I 
have allocated min 1G and max 2G heap size for it. How can i control the 
virtual memory usage? I am getting OOM frequently on ElasticSearch log.

  PID USER  PR  NI  VIRT  RES  SHR   S   %CPU %MEMTIME+  COMMAND
 3721 elastics  200  87.4g   2.1g 119m  S   66.214.3 
29:45.90 java

Thanks
Gokul

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4fbb3426-f8d3-4cee-a6d6-d0d8003fd8b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

how to use cluster.routing.allocation.allow_rebalance ?

[Gokul]

Hi, 
I am trying to ensure that re-allocation of shards are prevented whenever
one of the nodes in the cluster goes down. Based on the documentation I read
online, I use the following settings on the cluster

"persistent": {
"cluster.routing.allocation.enable": "none",
"cluster.routing.allocation.disable_allocation":"true"
}

However, this prevents new indices from being allocated as well. This is not
OK for me and so my requirement is to be able to do both the following
things - 
1. Be able to create indices dynamically. 
2. Prevent re-allocation of shards when a node in the cluster goes down. 

Based on the documentation, I understood that setting
cluster.routing.allocation.allow_rebalance to "indices_all_active" will do
both the things. But this didn't happen. Hence, my understanding is clearly
wrong. So what does it mean to set allow_rebalance to indices_all_active ?
When going over the mailing list, I also found the concept of "dynamic
settings". I assume this means that there are some values that I can set
when the cluster is live and some settings that I can only set in the
configuration file, which will be set on restarting elastic search. Is
allow_rebalance one such setting? 
Please correct me if I am on the wrong path altogether.

I see that I have raised multiple questions in one thread. However, all of
them are a related use case to me. Can you please advice me on these. 

Thanks,



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/how-to-use-cluster-routing-allocation-allow-rebalance-tp4061785.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1407924508318-4061785.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marvel Indices taking lot of space ? Can we specify automatic delete of marvel indice ?

[Gokul A]

Deepak, 

Can you try the command with a hyphen at the end -   
/usr/lib/python2.6/site-packages/curator/curator.py  -d 3 -p .marvel-
For some reason, curator doesn't match the marvel indices when using 
".marvel" as the prefix.




Regards,


On Thursday, May 15, 2014 1:14:14 AM UTC+8, Deepak Subhramanian wrote:
>
> Hi Ivan/Boaz,
>
> I am getting some warning when I tried to delete the marvel index using 
> curator. 
>
> github]#  /usr/lib/python2.6/site-packages/curator/curator.py  -d 3 -p 
> .marvel
> 2014-05-14T18:07:44.691 INFOmain:333  Job 
> starting...
> 2014-05-14T18:07:44.692 INFO   _new_conn:180  Starting new 
> HTTP connection (1): localhost
> 2014-05-14T18:07:44.694 INFO log_request_success:49   GET 
> http://localhost:9200/ [status:200 request:0.003s]
> 2014-05-14T18:07:44.695 INFOmain:359  Deleting 
> indices older than 3 days...
> 2014-05-14T18:07:44.698 INFO log_request_success:49   GET 
> http://localhost:9200/.marvel*/_settings?expand_wildcards=closed[status:200 
> request:0.002s]
> 2014-05-14T18:07:44.722 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.07
> 2014-05-14T18:07:44.722 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.08
> 2014-05-14T18:07:44.722 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.09
> 2014-05-14T18:07:44.722 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.10
> 2014-05-14T18:07:44.722 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.11
> 2014-05-14T18:07:44.723 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.12
> 2014-05-14T18:07:44.723 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.13
> 2014-05-14T18:07:44.723 ERROR   find_expired_indices:201  Could not 
> find a valid timestamp from the index: .marvel-2014.05.14
> 2014-05-14T18:07:44.723 INFO  index_loop:309  DELETE index 
> operations completed.
> 2014-05-14T18:07:44.723 INFOmain:379  Done in 
> 0:00:00.037963.
> [root@node05 github]# ls -ltr 
> /var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel*
> /var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.10:
> total 8
> drwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 10 01:00 _state
> drwxr-xr-x. 5 elasticsearch elasticsearch 4096 May 10 01:00 0
>
> /var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.11:
>
> Any thoughts?
>
>
> On Tue, May 13, 2014 at 12:26 PM, deepakas [via ElasticSearch Users] <[hidden 
> email] > wrote:
>
>> Hi Ivan/Boaz, 
>>
>> Thanks for the responses. I will checkout Curator. I already got the 
>> latest version of marvel last week using this command. 
>> bin/plugin -i elasticsearch/marvel/latest 
>>
>> Thanks, Deepak 
>>
>> --
>>  If you reply to this email, your message will be added to the 
>> discussion below:
>>
>> http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729p4055784.html
>>  
>>  To unsubscribe from Marvel Indices taking lot of space ? Can we specify 
>> automatic delete of marvel indice ?, click here.
>> NAML
>>  
>>
>
>
>
> -- 
> Deepak Subhramanian 
>
> --
> View this message in context: Re: Marvel Indices taking lot of space ? 
> Can we specify automatic delete of marvel indice 
> ?
> Sent from the ElasticSearch Users mailing list 
> archiveat Nabble.com.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/36baa5ec-c467-47df-b597-0aee612dd888%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.