Re: Elasticsearch shuts down for no reason

2014-08-24 Thread Eitan Vesely
Thanks Mark, auth.log doesnt show any login or sudo at the time of the elastic stopping... nothing else is running on that machine - it is a dedicated ES server. what i did find in the auth log is that someone is trying to hack into the system, yet i dont see how it got to do with elastic

Re: Elasticsearch shuts down for no reason

2014-08-24 Thread Eitan Vesely
I did find the shutdown request in the syslog: Aug 23 16:49:01 medisafelog2 kernel: [3361057.489168] hv_utils: Shutdown request received - graceful shutdown initiated yet i have no idea who or what initiated it... how can i dig in? On Sunday, August 24, 2014 12:01:50 PM UTC+3, Eitan Vesely

Re: Elasticsearch shuts down for no reason

2014-08-24 Thread joergpra...@gmail.com
The company which is providing the hosting service had shut down the virtual machine, hv_utils is a message from the hypervisor. This is not related to Elasticsearch at all. Jörg On Sun, Aug 24, 2014 at 11:19 AM, Mark Walkom ma...@campaignmonitor.com wrote: What version of ES are you

date_histogram facet float possible overflow

2014-08-24 Thread moshe zada
HI all, I am using ELK stack to visualising our monitoring data, yesterday i came across a weird problem: ElasticSearch date_histogram facet returned floating results that look like an overflow (min : 4.604480259023595*E* 18). Our dataflow is : collectd (cpu/memory) - sends it to riemann -

Re: Json Data not getting parsed when sent to Elasticsearch

2014-08-24 Thread moshe zada
what is your logstash configuration? did you tried the json codec http://logstash.net/docs/1.4.2/codecs/json? On Sunday, August 24, 2014 4:54:08 PM UTC+3, Didjit wrote: Hi, The following is a debug from Logstash: { message = {\EventTime\:\2014-08-24T09:44:46-0400\,\URI\:\

Re: Optimizing queries for a 5 node cluster with 250 M documents (causes OutOfMemory exceptions and GC pauses)

2014-08-24 Thread Jonathan Foy
I ran into the same issue when using Integer.MAX_VALUE as the size parameter (migrating from a DB-based search). Perhaps someone can come up with a proper reference, I cannot, but according to a comment in this SO

Re: Json Data not getting parsed when sent to Elasticsearch

2014-08-24 Thread Didjit
Pretty simple (below). . I just added to json codec and tried again and received the same results. Thank you! elasticsearch { host = localhost cluster = cjceswin node_name = cjcnode codec = json index = logstash-dwhse-%{+.MM.dd} workers = 3 } } On Sunday, August 24, 2014 10:11:44 AM

Re: Optimizing queries for a 5 node cluster with 250 M documents (causes OutOfMemory exceptions and GC pauses)

2014-08-24 Thread joergpra...@gmail.com
Exactly. Filters do not use scores. They also use bit sets which makes them reusable and fast. I wasn't talking about a filter added to a query, I mean filtered queries. This is a huge difference. This query { query : { bool: { must: { match : { body : big }

indices.memory.index_buffer_size

2014-08-24 Thread Yongtao You
Hi, Is the indices.memory.index_buffer_size configuration a cluster wide configuration or per node configuration? Do I need to set it on every node? Or just the master (eligible) node? Thanks. Yongtao -- You received this message because you are subscribed to the Google Groups elasticsearch

Re: Topics/Entities with relevancy scores and searching

2014-08-24 Thread Scott Decker
Interesting. so, set a payload on the term, in this case the topic/entity, and the payload is the relevancy value. Then, you can do your function score on the query of the main documents themselves, no need for parent/child. Have you done this? any concerns to performance with this sort of

Re: Boost the first word in a multi-word query

2014-08-24 Thread Jérémy
Thanks Vineeth, I can certainly build something with the query string :-) On Fri, Aug 22, 2014 at 8:50 PM, vineeth mohan vm.vineethmo...@gmail.com wrote: Hello Jeremy , You can try query_string then. Query as Brown^2 dog

Re: One large index vs. many smaller indexes

2014-08-24 Thread Chris Neal
Adrien, Thanks so much for the response. It was very helpful. I will check out those links on capacity planning for sure. One followup question. You mention that tens of shards per node would be ok. Are you meaning tens of shards from tens of indexes? Or tens of shards for a single index?

Re: indices.memory.index_buffer_size

2014-08-24 Thread Mark Walkom
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-indices.html states It is a global setting that bubbles down to all the different shards allocated on a specific node. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web:

Re: What fields does ElasticSearch map by default?

2014-08-24 Thread vineeth mohan
Hello Albert , Few things here 1. Yes , you cal tell Elasticsearch which fileds to index and which field not to index. You can use index : yes/no property for each field in the schema to specify this. -

Elasticsearch Function Score not working with object type

2014-08-24 Thread Pablo Musa
Hey guys, I am trying to use the function score but I am getting the following error: ElasticsearchIllegalArgumentException[No field found for [fsot] in mapping with types [tst]]; I have used function score before and it worked like a charm so I started digging what was wrong. I found out that

Re: DOS attack Elasticsearch with Mappings

2014-08-24 Thread Nikolas Everett
If the cluster is that open to users I don't think it'd be easy to prevent a malicious user from intentionally DOSing it. But in this case I think you could make the default for all fields be non-dynamic. That way users have to intentionally send all mapping updates. It'd prevent this short of

Re: Need some advice to build a log central.

2014-08-24 Thread vineeth mohan
Hello Sang , As this is a question answer forum , we highly recommend you to take a shot yourself and post questions if you have hit a dead end. Thanks Vineeth On Mon, Aug 25, 2014 at 7:56 AM, Sang Dang zkid...@gmail.com wrote: Hi All, I am going to build a log central using

Re: Elasticsearch Function Score not working with object type

2014-08-24 Thread vineeth mohan
Hello Pablo , Lucene ( the underlying library library on which ES is build upon) has only key value concept and it does not keep object level information. This means that on Lucene side , data would be stored as fsot.testObjects : [ test1 , test2 ] And there is not field names as fsot on lucene

Re: Elasticsearch Function Score not working with object type

2014-08-24 Thread pablitomusa
It worked. Thank you very much. * copying the final code for future referece: POST test/tst/_search { query: { function_score: { boost_mode: replace, query: { filtered: { query: { match_all: {} }, filter: { exists:

Re: Error running ES DSL in hadoop mapreduce

2014-08-24 Thread Sona Samad
Hi Adrien, My elasticsearch version is : elasticsearch-1.2.1 The Maven dependency for hadoop: dependency groupIdorg.elasticsearch/groupId artifactIdelasticsearch-hadoop-mr/artifactId version2.0.1/version /dependency The full stack trace is given below: [2014-08-25