Re: High cpu load but low memory usage.

Dear Kimchy, Plz suggest me what the problem could be ? Thank you so much ! Atrus. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@

hardware recommendation for dedicated client node

hi there, i wonder what is the hardware recommendation for the dedicated client node? i know master is a very light weight node that doesn't require good hardware, but how about client? it's saying the client node is doing the actual gather processing, so i assume it might require more memory l

Re: how to retrieve cluster and node stats on data node when disable http (http.enabled: false)

that's a good trick! thanks Otis! On Thursday, October 23, 2014 8:27:37 PM UTC-7, Otis Gospodnetic wrote: > > Hello, > > This works for SPM : > https://sematext.atlassian.net/wiki/display/PUBSPM/SPM+FAQ#SPMFAQ-CanSPMcollectmetricsevenwhenElasticsearchHTTPAPIisdisabled?

Elasticsearch thinks deleted repo snapshots are still available

We collect Twitter data every hour in separate indices. During testing of the snapshot facility in Elasticsearch 1.3.2, I created a repo (called backup0) and a snapshot (called 2014-07-23) of a single index (called tweets-2014-07-23:16). Then I added a second index (called tweets-2014-07-23:

Re: Unexpected behavior from nested -> filter -> nested aggregation

"Lastly, it is not possible to “cross reference” between nested documents. One nested doc cannot “see” another nested doc’s properties. For example, you are not able to filter on “A.name” but facet on “B.age”. You can get around this by using `include_in_root`, which effectively copies the neste

Re: Can i elastic search as my primary store?

I'd wait for 1.4 before considering it. There are lots of stability improvements there. One thing to consider is that updates are quite costly compared to Mongo/MySQL whatever. Nik On Fri, Oct 24, 2014 at 6:34 PM, Zennet Wheatcroft wrote: > I have heard from the source, Do not use Elasticsear

Re: nested aggregation against key value pairs

Have you tried the usual sub-aggregations? It looks like it should do exactly what you want. If so, why does that not work? Can you include some sample data and queries you have tried so that we can index it and try your queries? "Bucketing aggregations can have sub-aggregations (bucketing or m

Re: Can i elastic search as my primary store?

I have heard from the source, Do not use Elasticsearch as a data store. But some people do and it works ok. I would recommend that you use the snapshot and restore features. And back up your json file data so you can re-index in case your index gets corrupted. And be careful upgrading, especiall

Re: Unexpected behavior from nested -> filter -> nested aggregation

I'm also running into this it is not what I expected. I tried parent/child and got the same behavior. I expect the filtering to narrow down the results with each filter. I filter on a child (or nested) that has property=p then go back to aggregate on the parent and I get all the results again a

elasticsearch.index.mapper.MapperParsingException: object mapping for [article] tried to parse as object, but got EOF, has a concrete value been provided to it?

Hello, i'm trying to index xml document in elasticsearch after convert it to json but i have this error Please, i need help, is there someone who has any idea. Thanks in advance. --- This's an example of json document: {"Nserver":{"State":"stopped","ErrorDescription":"Success","ErrorID":{"c

Aggregation buckets, with an additional key:value inside.

Hey everyone, These aggregations are working out great, but I need to return more than one value in the bucket so we can use them in our API. The basic idea is that we aggregate all of the category id's, but we also want the category_name to be included in that same bucket for ease of use. *

Re: Migration of 0.90.3 cluster to new cluster running 1.3.4

The plan to move from a 2 node to a 3 node cluster is as follows - backup your old data files (in case you want to go back, once upgraded, there is no way back) - shutdown old cluster - move the data file folder of the old cluster nodes to the new cluster nodes data folders. One node gets no dat

Can i elastic search as my primary store?

Is it a use case today to use elastic search as a primary store? basically using it similar to mongodb? is that a use case the product is moving towards or it is mostly just for search? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsub

Re: partial index replication causes data loss?

Hi Evan, As Jorg said (though I wouldn't make the replica count == node count a golden rule), if you have 2 copies of your data it means that you are resilient to one failure at the time. If another failure occurs while you are still recovering from the first, bad things may happen. That said,

Re: Heavy load on a small Elasticsearch cluster

Le vendredi 24 octobre 2014 19:59:01 UTC+2, Jörg Prante a écrit : > > You're doomed :) > > What monitoring tool do you use? Try to reduce the frequency at least. > > Jörg > > New Relic monitor the OS but don't touch ES. I used a patched version of Diamond. I unplugged it and I RTFM for specific

nested aggregation against key value pairs

I have an ES type with a nested KeyValuePair type. What I'm trying to do is a terms aggregation on both the key and value fields such that I'd get the following results: Key1 - Value1: DocCount = 10 Key1 - Value2: DocCount = 9 Key2 - Value3: DocCount = 4 Here is my mapping: { "index123" : { "m

Re: Migration of 0.90.3 cluster to new cluster running 1.3.4

Oh, didn't know about optimize so I'll definitely keep that in mind. The reason I was asking about primary shards is that I saw, when starting from a rsync'd datafolder off of one of the nodes, double the amount of documents. It wasn't immediatly apparent but when I later on tried with two rsyncs

Re: Heavy load on a small Elasticsearch cluster

You're doomed :) What monitoring tool do you use? Try to reduce the frequency at least. Jörg On Fri, Oct 24, 2014 at 6:30 PM, Mathieu Lecarme wrote: > A simple cluster, 2 nodes, replica 1. Each node has 1.5Go RAM, 2 cores, > SAS disks. > With Elasticsearch 1.1 some deconnection appears, and so

Re: partial index replication causes data loss?

Interesting @Jörg How many nodes would you need then to not replicate all data on all nodes? A highly-touted feature of ES is the ability to share and spread data across nodes. Any recommendations? -- Evan Tahler | evantah...@gmail.com | 412.897.6361 evantahler.com | actionherojs.com On Fri, O

Heavy load on a small Elasticsearch cluster

A simple cluster, 2 nodes, replica 1. Each node has 1.5Go RAM, 2 cores, SAS disks. With Elasticsearch 1.1 some deconnection appears, and some CPU load picks. Bad usage of logstash (lots of tiny bulk imports, with monthly indices). Logstash usage was fixed, and Elasticsearch upgraded to 1.3. 1.3.3,

Re: Migration of 0.90.3 cluster to new cluster running 1.3.4

Unless you are moving to new hardware, there is no need to rsync your data. Both Elasticsaerch 0.90.x and 1.3.x are based on Lucene 4, so the underlying data is compatible. Of course, you should backup your data before such an upgrade. After restarting your new cluster with your old data, I would

Indexing search results into a seperate index

Hello All: I have a situation wherein, I would like to index a search result (from a search query1), However I want to index only a subset of the content from each document that is returned after some processing of certain fields. (index2) Is it possible so I can actually do some further que

Re: Comparison of int/long and float/double types

It depends what you do with ints. Your question was about disk storage. Ints are much faster when they are loaded into cache: they save 50% memory, they can be used as index in array for sorting, loading/storing by CPU instruction takes only one cycle etc. Jörg On Fri, Oct 24, 2014 at 4:48 PM, T

Re: Term Aggregate on Analyzed Fields?

As far as I know, you can not do this, because ES do not store field value by default(It uses ‘_source’). If ES read the value from ‘_source’, it will cost a long time. > 在 2014年10月24日，下午10:32，Matt Hughes 写道： > > I have a field with values like: > > foo > bar > bar-one > > Unfortunately, wh

Re: Comparison of int/long and float/double types

So let's assume the cardinality is the same. Let's assume I have no text, I only index numeric fields. If I've got a range of data, that would all fit within the bounds of an integer, is there any reason not to index it as a long? Are there any down sides? It sounds like you're saying that ther

Re: How to collect docs in order in collect() method of a custom aggregator

I asked the same question on github and got the answer which worked for me. Here's the solution: You can call aggregationContext.ensureScoreDocsInOrder(); to make sure that docs are going to come in order, have a look for instance at ReverseNestedAggregator which uses this method. Queries ar

Term Aggregate on Analyzed Fields?

I have a field with values like: foo bar bar-one Unfortunately, when I set up this index, I didn't realize that I wanted to turn off tokenization ("index": "not_analyzed"). Now when I try and do terms aggregation, I get back the tokenized values: foo bar (2) one Is there any way to do an ag

Re: partial index replication causes data loss?

If you have replica level 1 with 3 nodes, this is not enough. You must set replica level 2. With replica level 1 and outage of 2 nodes, as you describe, you will lose data. Jörg On Wednesday, October 15, 2014 12:52:31 AM UTC+2, Evan Tahler wrote: > > Hi Mailing List! I'm a first-time poster, a

Re: Help: Flattened aggregations (with limiting and sorting)

Thoughts, anybody? I saw that you can somewhat do this with "scripts" and letting the top aggregation encompass all term fields, but is that any more performant? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group

Re: Comparison of int/long and float/double types

Lucene only knows how to index text strings. For numeric types, they are stored as tries. Tries work on variable length. So only the API is different to convert integer or long to tries. Tries are the basis for numeric range searches. It is a myth that long take more disk space than ints in an inv

Comparison of int/long and float/double types

I get the impression that using the 'long' type instead of 'integer' would use more disk space and degrade search performance (similary for double instead of float), but there's nothing in the documentation to back this impression up. There must be an advantage to using integer (if you can) bec

[ANN] Elasticsearch RabbitMQ River plugin 2.4.0 released

Hello, We are pleased to announce the release of the Elasticsearch RabbitMQ River plugin, version 2.4.0. The RabbitMQ River plugin allows index bulk format messages into elasticsearch. https://github.com/elasticsearch/elasticsearch-river-rabbitmq/ Release Notes - elasticsearch-river-rabbitmq

Find the 100 closest neighbors to a point (lng, lat)

Hello, sorry if this question has already been asked, but I didn't find much material during my search. Basically, what I need to do, is find the exact 100 closest objects around a spatial point (longitude, latitude), among a 1 million geolocalized object all around the world. Is there an effi

High cpu load but low memory usage.

Hi Pros, I've used ES for several months, it works perfectly and speed as lightning. There are 3 nodes in cluster, each have 12cores CPU and 24GB-32GB RAM. For some recent days, the cpu get too h

ES 1.3.4: sometimes plugins return empty page

Some days ago I updated my 2-node ES Cluster to 1.3.4. Since the update the installed plugins sometimes are not applicable and for http://localhost:9200/_plugin/head returns an empty page. This effect is seen very often after restarting a node, e.g. after SHUTDOWN or simply killing the server p

Re: Future of cardinality aggregation feature.

Thanks a lot Adrien. On Sunday, October 19, 2014 1:54:22 PM UTC-7, Adrien Grand wrote: > > Hi Govind, > > I don't see this aggregation being removed in the future. The experimental > warning was rather about being able to change configuration options within > a minor version update (for example

Re: SSL reverse proxy in front of api on a cluster

The inter-cluster comms would still need secured too, I guess, though it is behind a firewall, so basic auth is less of an issue. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it,

SSL reverse proxy in front of api on a cluster

Hello. Right now ive got an ES cluster, and I would like it to use SSL for security reasons (nodes located across the internet from one another, remote api requests) . I have considered using nginx as a reverse proxy (I would also like to do HTTP basic auth) - But im not quite sure how that wo

Re: OutOfMemory

Thanks for the help. I am getting errors like this, which i interpret as that execution of a query requested more than available memory. Caused by: java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.util.FixedBitSet.(FixedBitSet.java:54) at org.apache.lucene.search.M

Re: geo_shape field not being returned with explicit 'fields' query

Based on this , I suppose the behavior is 'as designed': "Due to the complex input structure and index representation of shapes, it is not currently possible to s