Thanks guys. The previous ES had not stopped fully, so killed it, then deleted the node/1 directory, then set the node.max_local_storage_nodes to 1 in yml. Now everything works smoothly!
I've also increased the heap space. Though I hope ES doesn't consume the entire 10g heap space! On Wednesday, March 25, 2015 at 1:41:16 AM UTC+5:30, Yogesh wrote: > > Thanks. Will try starting again. > By the way, what will be the impact if I just delete the node/1 (then the > node.max_local_storage_nodes to 1 and start elasticsearch) ? > > On Wed, Mar 25, 2015 at 1:36 AM, <mjdu...@gmail.com> wrote: > >> As long as it's the first ES instance starting on that node it'll grab 0 >> instead of 1. I don't know if you can explicitly set the data node >> directory in the config. >> >> On Tuesday, March 24, 2015 at 2:16:57 PM UTC-4, Yogesh wrote: >>> >>> Thanks. Mine is a one node cluster so I am simply using the XCURL to >>> shut it down and the doing "bin/elasticsearch -d" to start it up. >>> To check if it has shutdown, I try to hit it using http. So, now how do >>> I start it with the node/0 data directory? >>> There is nothing there in node/1 data directory but I don't suppose >>> deleting it would be the solution? (Sorry for the basic questions I am new >>> to this!) >>> >>> On Tue, Mar 24, 2015 at 11:32 PM, <mjd...@gmail.com> wrote: >>> >>>> Sometimes that happens when the new node starts up via monit or other >>>> automated thing before the old node is fully shutdown. I'd suggest >>>> shutting down the node and verify it's done via ps before allowing the new >>>> node to start up. In the case of monit if the check is hitting the http >>>> port then it'll think it's down before it actually fully quits. >>>> >>>> On Tuesday, March 24, 2015 at 1:56:54 PM UTC-4, Yogesh wrote: >>>>> >>>>> Thanks a lot mjdude! It does seem like it attached to the wrong data >>>>> directory. >>>>> In elasticsearch/data/tool/nodes there are two 0 and 1. My data is in >>>>> 0 but node stats shows the data directory as >>>>> elasticsearch/data/tool/nodes/ >>>>> 1. >>>>> Now, how do I change this? >>>>> >>>>> On Tue, Mar 24, 2015 at 11:02 PM, <mjd...@gmail.com> wrote: >>>>> >>>>>> When it restarted did it attach to the wrong data directory? Take a >>>>>> look at _nodes/_local/stats?pretty and check the 'data' directory >>>>>> location. Has the cluster recovered after the restart? Check >>>>>> _cluster/health?pretty as well. >>>>>> >>>>>> On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote: >>>>>>> >>>>>>> Thanks Joel and mjdude. What I mean is that ES is using 99% of the >>>>>>> heap memory (I think since Marvel showed memory as 1 GB which >>>>>>> corresponds >>>>>>> to the heap, my RAM is 50GB) >>>>>>> I've increased the ES_HEAP_SIZE to 10g. But another problem has >>>>>>> appeared and I'm freaked out because of it! >>>>>>> >>>>>>> So, after restarting my ES (curl shutdown) to increase the heap, the >>>>>>> Marvel has stopped showing me my data (it still shows that disk memory >>>>>>> is >>>>>>> lower so that means the data is still on disk) and upon searching Sense >>>>>>> shows "IndexMissingException[[my_new_twitter_river] missing]" >>>>>>> >>>>>>> Why is this happening?!?! >>>>>>> >>>>>>> On Mon, Mar 23, 2015 at 9:25 PM, <mjd...@gmail.com> wrote: >>>>>>> >>>>>>>> Are you saying JVM is using 99% of the system memory or 99% of the >>>>>>>> heap? If it's 99% of the available heap that's bad and you will have >>>>>>>> cluster instability. I suggest increasing your JVM heap size if you >>>>>>>> can, I >>>>>>>> can't find it right now but I remember a blog post that used twitter >>>>>>>> as a >>>>>>>> benchmark and they also could get to ~50M documents with the default >>>>>>>> 1G >>>>>>>> heap. >>>>>>>> >>>>>>>> On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I have set up elasticsearch on one node and am using the Twitter >>>>>>>>> river to index tweets. It has been going fine with almost 50M tweets >>>>>>>>> indexed so far in 13 days. >>>>>>>>> When I started indexing, the JVM usage (observed via Marvel) >>>>>>>>> hovered between 10-20%, then started remaining around 30-40% but for >>>>>>>>> the >>>>>>>>> past 3-4 days it has continuously been above 90%, reaching 99% at >>>>>>>>> times! >>>>>>>>> I restarted elasticsearch thinking it might get resolved but as >>>>>>>>> soon as I switched it back on, the JVM usage went back to 90%. >>>>>>>>> >>>>>>>>> Why is this happening and how can I remedy it? (The JVM memory is >>>>>>>>> the default 990.75MB) >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Yogesh >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>> the Google Groups "elasticsearch" group. >>>>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>>>>>> topic/elasticsearch/kMSwBqpe2N4/unsubscribe. >>>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>>> elasticsearc...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/elasticsearch/0f990291-fa2 >>>>>>>> 0-4bba-881f-3f378985c8c9%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "elasticsearch" group. >>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/to >>>>>> pic/elasticsearch/kMSwBqpe2N4/unsubscribe. >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> elasticsearc...@googlegroups.com. >>>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>>> msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40goo >>>>>> glegroups.com >>>>>> <https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "elasticsearch" group. >>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>> topic/elasticsearch/kMSwBqpe2N4/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> elasticsearc...@googlegroups.com. >>>> To view this discussion on the web visit https://groups.google.com/d/ >>>> msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075% >>>> 40googlegroups.com >>>> <https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> elasticsearch+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ba9ddf48-bde9-4da4-b852-b666b6842c3f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
