Cannot Read Elasticsearch date type with format basic_date_time in Spark

2015-05-11 Thread Nicolas Phung
Hello, I'm trying to build a RDD from ElasticSearch data with elasticsearch-spark 2.1.0.Beta4. I have the following field mapping : date: {type: date,format: basic_date_time}, But when I'm trying to read it with Spark via this snippet: val rdd = sc.esRDD(analytic/docs, ?q=*) rdd.take(10)

Upgrading to Elastic 1.5.1 from 1.4.2 started raising ClassCastException

2015-05-11 Thread Eduardo Castegnaro
Hi there! I have a small cluster that was running fine on Elastic 1.4.2 and due to the stability improvements mentioned on 1.5 we decided to upgrade. The upgrade was done as a rolling upgrade (one instance at a time) and everything appeared to be going fine. We don't use Logstash, but we do

scripting contains integer versus long

2015-05-11 Thread eunever32
I have a mvel script (groovy looks the same) as follows: if (!ctx._source.list.contains(document)) {ctx._source.list += document;} else {ctx.op = \none\}; I have a java map map.put(id, 100L); When I try to match the above document to an existing ES document it will fail because it's a

Re: scripting contains integer versus long

2015-05-11 Thread Nikolas Everett
I suspect at that point they'll pop out as Longs. Its just my suspicion. I haven't read that bit of the code. On May 11, 2015 10:08 AM, euneve...@gmail.com wrote: I have a mvel script (groovy looks the same) as follows: if (!ctx._source.list.contains(document)) {ctx._source.list +=

Elasticsearch query for time the value has not changed

2015-05-11 Thread Aravind Chennuru
Hi All, I am kind of new to elasticsearch queries and I would like to get the information about the time since the last value change, in the following example the time the value is 1 = 5 seconds. Is there a way to do it, if no, can someone think of a way to achieve this? @timestamp: May 11th

Re: Forums Are Now Live at http://discuss.elastic.co

2015-05-11 Thread joergpra...@gmail.com
+1 for Doug and Ivan I'd also like to find the real names (which are available at Discuss because they are shown in the profile of a user) being added to the message view and the mail From header, for a more personal communication style. It would be easier to begin a reply with a greeting then.

sort missing parameter from query string

2015-05-11 Thread sebastian
Hello, Can I specify the sort missing parameter from query string? From body: { sort : [ { price : {missing : _last} }, ], query : { term : { user : kimchy } } } Thanks! -- Please update your bookmarks! We have moved to https://discuss.elastic.co/ --- You

Re: sort missing parameter from query string

2015-05-11 Thread sebastian
Is it possible to specify it from mapping? On Monday, May 11, 2015 at 5:05:27 PM UTC-3, sebastian wrote: Hello, Can I specify the sort missing parameter from query string? From body: { sort : [ { price : {missing : _last} }, ], query : { term : { user :

Re: Elasticsearch hardware requirement,and benchmarking

2015-05-11 Thread haries fajar nugroho
Hi Mark, Does data volume that you mean is my current log files that need to be processed ? If yes, it is 1.5-3 gb/hour. If index rate that you mean is Indexing request per second in bigdesk, it is 74839326 /seconds. If search rate means search request per second in bigdesk, it is 500-700

Re: Error handling for HTTP search API

2015-05-11 Thread Snixtor
So I'm still none-the-wiser. It feels like this is a notable omission in the Elasticsearch implementation. I *want* to implement more graceful error handling for my Elasticsearch integration, but the means don't seem to be available. Is this the case? The deeper I consider the situation, the

Kibana never fully loads, stays searching

2015-05-11 Thread Tony Chong
Hi, Strange issue. Kibana and indexing works correctly at first but after several weeks, the UI always shows that Kibana is searching. No matter what I try to query or time interval, it's always searching and never returns anything. I don't see anything funky in the logs and if I delete all

Re: Elasticsearch hardware requirement,and benchmarking

2015-05-11 Thread Mark Walkom
What about data volumes, index and search rates? On 11 May 2015 at 11:40, haries fajar nugroho harie...@gmail.com wrote: Hi Mark, Thanks, really appreciate your response. Previously i thought that it is better kibana to communicate with master rather than data node, but i was wrong. So

Re: Troubleshooting ES Resharding. Nature of immediate tasks and other questions

2015-05-11 Thread Mark Walkom
If your largest index is only 1GB, then try reducing that to a single shard (with one replica) and see how the performance goes. I'd trial that on a few indices before rolling it out across the board though. Again, the core issue is that you have too many shards! On 7 May 2015 at 00:34,

Perform Union on Two Sorted Queries

2015-05-11 Thread wajahat
Hi all, I have a use case where I have to run two elastic search Queries which are paginated, have a sorting criteria, but the number of results are same, I would like to know if there is a possibility to UNION both queries, besides doing it my self by iterating results of both the queries at