jdk fails with out of memory error / es critical index counts

2014-05-04 Thread Nish
elasticsearch is set as a single node instance on a 60G RAM and 32*2.6GHz machine. I am actively indexing historic data with logstash. It worked well with ~300 million documents (search and indexing were doing ok) , but all of a sudden es fails to starts and keep itself up. It starts for few

Re: Elasticsearch with JSON-array, causing serialize -error

2014-05-04 Thread PyrK
Anyone have any suggenstions how to prevent this? sunnuntai, 20. huhtikuuta 2014 20.47.40 UTC+3 PyrK kirjoitti: I'm using elasticsearch with mongodb -collection using elmongohttps://github.com/usesold/elmongo. I have a collection (elasticsearch index's point of view json-array), that

Nested Aggregations

2014-05-04 Thread Niv Penso
Hey guys, I have this mappings: { ckdocuments: { mappings: { msv: { properties: { MSV: { type: nested, properties: { country: { type: string,

Re: Nested Aggregations

2014-05-04 Thread Adrien Grand
Oops, I missed to include the link: [1] http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-nested-aggregation.html On Sun, May 4, 2014 at 7:20 PM, Adrien Grand adrien.gr...@elasticsearch.com wrote: Hi Niv, MSV is a nested field. Nested fields

Re: Nested Aggregations

2014-05-04 Thread Adrien Grand
Hi Niv, MSV is a nested field. Nested fields are effectively stored in separate documents, you need to use the nested[1] aggregation in order to make this aggregation work. It would look something like (not tested): aggregations:{ MSV: { nested: { path: MSV }, aggregations: {

[Hadoop] Writing directly to shards in EsOutputFormat and shard awareness

2014-05-04 Thread Ashwin Jayaprakash
Hi, I have 2 related questions regarding routing write requests. Thanks in advance for answering! *Question 1:* I saw this line in the EsOutputFormat class and I was wondering why:

Re: [Hadoop] Writing directly to shards in EsOutputFormat and shard awareness

2014-05-04 Thread Costin Leau
n 1. Performance reasons While a write request can be send to any node, which in turn will do proxying, we can avoid this and only hit the primaries. This avoids the proxying, rerouting. Note that each task that is writing is assigned a different primary in a round-robin fashion - so effectively

Re: calculations in KIBANA

2014-05-04 Thread Patrick Proniewski
Hello again, Anyone on this? Thanks, Patrick On 01 mai 2014, at 15:33, Patrick Proniewski wrote: Hello, I understand that Kibana is not ES, but they are closely related, so I give my question a try here: I would like to be able to process numbers after query. For example, I'm using

what is my actual index_buffer_size?

2014-05-04 Thread R. Toma
Hi group, Anyone know how to query the actuel index_buffer_size? I have searched thru the _cluster/stats output, but cannot find it. Using 3-node ES 1.0.1 for a logsearch platform (lots of logstash bulking, less kibana querying) with fresh indices every day, I have

Re: what is my actual index_buffer_size?

2014-05-04 Thread joergpra...@gmail.com
You can not measure the actual index_buffer_size by API. It is a dynamic setting, ES adapts its size regularly in the background, depending on indexing activity. The number is a maximum estimated value. 50% is a good value for bulk indexing in comparison to the default 10%. It means, indexing

Re: Help with ES 1.x percolator query plz

2014-05-04 Thread JGL
Hi Martjin, The percolator query in the 1st post above is what we registered to the percolator and kinda working, which consolidate all IDs in one query string for a match query, which seems not quite a elegant solution to us. { _index : my_idx, _type : .percolator, _id :

Fail to put dynamic mapping after upgrading from 0.90.1. to 1.1.1

2014-05-04 Thread Hui
Hi All, In elasticsearch 0.90.1, we do the following : curl -XPUT 'http://localhost:9200/testing/' -d '{ settings : { index : { number_of_shards : 1, number_of_replicas : 0, refresh_interval : 1m } } }' curl -XPUT

MoreLikeThis can't identify that 2 documents with exactly same attachments are duplicates

2014-05-04 Thread Zoran Jeremic
Hi guys, I have a document that stores a content of html file, pdf, doc or other textual document in one of it's fields as byte array using attachment plugin. Mapping is as follows: { document:{ properties:{ title:{type:string,store:true },

ANN: new elasticsearch discovery plugin - eskka

2014-05-04 Thread shikhar
See README https://github.com/shikhar/eskka/blob/release-0.1/README.md I'd love to have feedback on this first release! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an