Re: Need help/suggestion for the massive queries user case

Yes, This works fine. The query request will eventually get processed. It just throws this message. Thanks. On Tuesday, September 2, 2014 4:51:08 PM UTC-4, David Pilato wrote: > > I have no idea. I don't know the Perl Client. > > Does this works? > > curl localhost:9200 > > > > -- > *David Pi

Re: Need help/suggestion for the massive queries user case

I have no idea. I don't know the Perl Client. Does this works? curl localhost:9200  --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 2 septembre 2014 à 22:46:15, Yuheng Du (yuheng.du.h...@gmail.com) a écrit: Tried to increase the memory for the VM, n

Re: Need help/suggestion for the massive queries user case

Tried to increase the memory for the VM, now ES runs ok, but still gives me error: Which is caused by the Elasticsearch's perl module's Try::Tiny module. How to remove this kind of warning/err

Re: Need help/suggestion for the massive queries user case

You don't have enough memory available on your laptop I guess... -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 2 sept. 2014 à 21:05, Yuheng Du a écrit : Ok. you are right. I was able to set the maximum heap size and now it looks like: "jvm" : { "timestamp" : 140968439

Re: Need help/suggestion for the massive queries user case

Ok. you are right. I was able to set the maximum heap size and now it looks like: "jvm" : { "timestamp" : 1409684394790, "uptime_in_millis" : 128401, "mem" : { "heap_used_in_bytes" : 520385872, "heap_used_percent" : 24, "heap_committed_in_bytes" : 2138767360, "heap_max_in_bytes" : 21387673

Re: Need help/suggestion for the massive queries user case

I think your JVM settings are incorrect as we can see with heap_max -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 2 sept. 2014 à 20:37, Yuheng Du a écrit : BTW, the node stats info shows the jvm usage info as follows: "jvm" : { "timestamp" : 1409682903048, "uptime_in_mi

Re: Need help/suggestion for the massive queries user case

BTW, the node stats info shows the jvm usage info as follows: "jvm" : { "timestamp" : 1409682903048, "uptime_in_millis" : 488996, "mem" : { "heap_used_in_bytes" : 48476472, "heap_used_percent" : 9, "heap_committed_in_bytes" : 73412608, "heap_max_in_bytes" : 515571712, "non_heap_used_in_bytes" : 570

Re: Need help/suggestion for the massive queries user case

Ok. The shards are recovered now. I tried to set the ES_HEAP_SIZE to 2g, but I am still getting "Nodes not available" error: What can be the cause then? Thanks! Yuheng On Tuesday, September

Re: Need help/suggestion for the massive queries user case

You have to start again the second node on the same machine to get the other shards back. Then when you cluster is yellow again, you can switch off the second node. BTW, you should set ES_HEAP_SIZE instead of dealing with Xmx Xms (but this will do the same thing behind the scene). See  http://w

Re: Need help/suggestion for the massive queries user case

I tired 2g, It only worked when I use ./elasticsearch -Xmx2g -Xms2g. Currently some of my shards are stated as "unassigned" in my kopf web interface. It happened after I started 3 nodes at the same time in my desktop yesterday. There should be 3G of data in the cluster, but know I see only 1.92

Re: Need help/suggestion for the massive queries user case

Try 2g -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 2 sept. 2014 à 15:49, Yuheng Du a écrit : > > I tried to run ./elasticsearch --node.name=esmaster -Xmx=2G -Xms=2G. > But it gives me the error msg: Invalid maximum heap size: -Xmx=2G, > > How can I deal with this?

Re: Need help/suggestion for the massive queries user case

I tried to run ./elasticsearch --node.name=esmaster -Xmx=2G -Xms=2G. But it gives me the error msg: Invalid maximum heap size: -Xmx=2G, How can I deal with this? Thanks On Tuesday, September 2, 2014 3:46:26 AM UTC-4, David Pilato wrote: > > Ha! On your desktop? So that's not a production issue.

Re: Need help/suggestion for the massive queries user case

Ha! On your desktop? So that's not a production issue. In that case, you should may be give more memory to all your elasticsearch nodes. What is your current HEAP size? --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 2 septembre 2014 à 09:17:31, Yuheng

Re: Need help/suggestion for the massive queries user case

Each query is doing a search in _all indexes in the cluster. Maybe that's also a hold up of the availability of ES server. I tried to run 3 ES nodes on my desktop machine but it does not seems to help, it also crashed after some time. On Tuesday, September 2, 2014 3:10:13 AM UTC-4, David Pilat

Re: Need help/suggestion for the massive queries user case

May be you could increase the number of replicas and nodes so your cluster could handle more queries at a time. The main cost is of course in term of machine and disk space. Can't tell really more without more details about what you are doing exactly. May be others have some ideas? --  David Pi

Need help/suggestion for the massive queries user case

Hi guys, I have some streaming sensor data as input to ES. For each incoming data message, I need to do a query on the historic data in ES according to the 'timestamp' and 'messageId' in that message. I need to get the aggregated query results in real-time. My problem is each data message may