Re: Aggregrations - Filter on (nested value OR nested value) AND non nested value

2015-01-09 Thread Dev Day
Hi All, I think I've figured it out: filter: { and: { or: [ { nested: { path: variants, filter: {

Re: conditional function score query.

2015-01-09 Thread joergpra...@gmail.com
You can check out https://github.com/jprante/elasticsearch-functionscore-conditionalboost for an example of how this can be done with a Java plugin. Jörg On Fri, Jan 9, 2015 at 11:38 AM, Jason Wee peich...@gmail.com wrote: Hi, I have build a query using java FunctionScoreQueryBuilder

Re: Concurrency problem when automatically creating an index

2015-01-09 Thread joergpra...@gmail.com
Exactly, with 3 nodes, the error will be gone. Please, always use an odd number of data nodes, in particular with replica 0, in order not to confuse ES quorum formula, and also to avoid split brains with minimun_master_nodes Jörg On Fri, Jan 9, 2015 at 3:17 PM, Tom tarossi.despe...@gmail.com

Aggregrations - Filter on (nested value OR nested value) AND non nested value

2015-01-09 Thread Dev Day
Hi All, Please see a working example of a document. The document has variants as a nested document within each root document. The data has 3 records as can be seen below. I would like to do the following: 1) For the filter aggregation for colour, I would like to filter by two ranges, in a OR

Re: Upgraded node unable to join cluster while attempting cluster upgrade from 1.3.2 to 1.4.2

2015-01-09 Thread Radu Gheorghe
Thanks Ben and Joerg. Too bad there was no good solution this particular upgrade, at least we all learned something (well, aside from Joerg who knew it all from the start :D). Best regards, Radu -- Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch Support *

Re: corruption when indexing large number of documents (4 billion+)

2015-01-09 Thread Mark Walkom
Honestly, with this sort of scale you should be thinking about support (disclaimer: I work for Elasticsearch support). However let's see what we can do; What version of ES, java? What are you using to monitor your cluster? How many GB is that index? Is it in one massive index? How many GB in your

Re: High CPU load when using Logstash file input on Windows

2015-01-09 Thread Vagif Abilov
Oops, looks like Logstash has its own forum. Will try there. Vagif On Fri, Jan 9, 2015 at 10:06 AM, Vagif Abilov vagif.abi...@gmail.com wrote: Hello, We started using ELK stack to visuallize our logs, and we observe high CPU load on machines (Windows servers) where text logs are generated.

High CPU load when using Logstash file input on Windows

2015-01-09 Thread Vagif Abilov
Hello, We started using ELK stack to visuallize our logs, and we observe high CPU load on machines (Windows servers) where text logs are generated. There aren't many log files but they can be very large in size. They contain logs formatted as json documents so there shouldn't be much computation

Re: Upgraded node unable to join cluster while attempting cluster upgrade from 1.3.2 to 1.4.2

2015-01-09 Thread joergpra...@gmail.com
I could switch away from plugin mode, but that would mean people must adapt and sell another JDBC bridge component to their boss and worse, that will keep the SPOF weakness of rivers but with more burden of responsibility on myself. Not in my sparetime. Maybe a compromise will work better:

Re: Upgraded node unable to join cluster while attempting cluster upgrade from 1.3.2 to 1.4.2

2015-01-09 Thread joergpra...@gmail.com
No I didn't :( I still struggle with stateful plugins - there is no good solution I know of and that is the reason of trouble. Jörg On Fri, Jan 9, 2015 at 9:01 AM, Radu Gheorghe radu.gheor...@sematext.com wrote: Thanks Ben and Joerg. Too bad there was no good solution this particular

Re: Concurrency problem when automatically creating an index

2015-01-09 Thread joergpra...@gmail.com
It seems there are more than one process trying to create the index, it that possible? Jörg On Fri, Jan 9, 2015 at 7:16 PM, Tomas Andres Rossi tro...@despegar.com wrote: We enlarged our cluster to 5 nodes and now the QUORUM error message seems to have disappeared. failed to process cluster

Re: Concurrency problem when automatically creating an index

2015-01-09 Thread Tom
Sorry, didn't mean to say the same indexing operation but multiple indexing operations (distinct data) on the same non-existent index. El viernes, 9 de enero de 2015, 16:13:52 (UTC-3), Tom escribió: Well yes. We also have a cluster for the app where each node talks to the elastic cluster

Re: Concurrency problem when automatically creating an index

2015-01-09 Thread Tomas Andres Rossi
We enlarged our cluster to 5 nodes and now the QUORUM error message seems to have disappeared. failed to process cluster event (acquire index lock) within 1s kind of messages are still happening though. :( Tom; On Fri, Jan 9, 2015 at 3:11 PM, Tomas Andres Rossi tro...@despegar.com wrote: We

Re: Searching with Elasticsearch.Net

2015-01-09 Thread Itamar Syn-Hershko
If all you need is querying, I will highly recommend looking at https://github.com/CenturyLinkCloud/ElasticLINQ for .NET I also have my own stab at a .NET client library for Elasticsearch here: https://github.com/synhershko/NElasticsearch / https://www.nuget.org/packages/NElasticsearch/1.0.14

Re: corruption when indexing large number of documents (4 billion+)

2015-01-09 Thread Darshat Shah
Hi Mark, Thanks for the reply. I need to prototype and demonstrate at this scale first to ensure feasibility. Once we've proven ES works for this use case then its quite possible that we'd engage with support for production. Regarding your questions: *What version of ES, java? *

Kibana dashboards not saving for me in ElasticSearch

2015-01-09 Thread William Tarrant
Hey, after a long day with no success I finally conceded that I need to ask for help. I am running ES 1.4 with kibana 3.1.2 hosted on ngnix both on the same machine (I have done the cors work around). Whenever I try to save a new dashboard or retrieve it, it results in a blank page, it

Re: Indices are missing. Help!

2015-01-09 Thread Matthew Eash
As a followup - seems there is a major issue in indices created from a mapping template. I installed a fresh copy of ES 1.4.2 standalone on my laptop and replicated the issue I had on my ES 1.4.1 cluster -- disappearing indices on cluster restart.

Re: Concurrency problem when automatically creating an index

2015-01-09 Thread Tom
Well yes. We also have a cluster for the app where each node talks to the elastic cluster independently. Remember that we are not creating the index manually. Each app node issues an index operation on an index that may yet not exist and we expect ES to take care of the index creation on

`store`'d fields w/ `index_name` in mapping loaded using `fields` returned in `hits` objects keyed by `index_name`?

2015-01-09 Thread Luke Nezda
In attempting to work around https://github.com/elasticsearch/elasticsearch/issues/5851, I found that `store`'d fields with `index_name` in their mapping loaded using ` fields http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html` are returned in

weighted average scripted metric usage

2015-01-09 Thread Kallin Nagelberg
The current 1.4 docs mention that the scripted_metric aggregation is experimental, and to share our usages. I've found a really great use on our project so I thought I'd share! While the 'stats' metric provides great data like 'sum' and 'average', we needed to calculate a weighted average. In

Search not working unless type specified

2015-01-09 Thread Stefanie
I am having an issue with searching results if the type is not specified. The following search request works correctly: curl -XGET 'http://localhost:9200/twitter/tweet/_search' -d '{ query : {match_all: {}}, filter: { terms: { tweet_id:[ 128 ] } }, sort :

Re: Kibana dashboards not saving for me in ElasticSearch

2015-01-09 Thread Mark Walkom
It'd help if you could gist/pastebin/etc your nginx and kibana configs. On 10 January 2015 at 07:57, William Tarrant tarrant.will...@gmail.com wrote: Tried a clean install with ES 1.3.7 and Kibana 3.1.2 and still the same issue. I must be missing something with permissions or nginx, as I

Re: Kibana dashboards not saving for me in ElasticSearch

2015-01-09 Thread William Tarrant
Tried a clean install with ES 1.3.7 and Kibana 3.1.2 and still the same issue. I must be missing something with permissions or nginx, as I tried the same combination on windows using python simple server and it worked. w On Friday, January 9, 2015 at 4:03:46 PM UTC, William Tarrant wrote:

Re: Upgraded node unable to join cluster while attempting cluster upgrade from 1.3.2 to 1.4.2

2015-01-09 Thread Radu Gheorghe
Yeah, my general approach so far was to stay away from plugins as much as possible and use external services instead. Something that your JDBC river can do as well, as far as I understand. -- Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch Support *

Re: Error - MapperParsingException-No handler for type [attachment] declared on field

2015-01-09 Thread David Pilato
Hi Shashi, I just tried to reproduce your issue. I: * downloaded elasticsearch 1.4.2 * ran bin/plugin -install elasticsearch/elasticsearch-mapper-attachments/2.4.1 * ran bin/elasticsearch Then: curl -XPUT http://localhost:9200/test http://localhost:9200/test curl -XPUT

Allow for disabled sorting on aggregations when request size is unlimited

2015-01-09 Thread Elliott Bradshaw
In cases where all aggregation results are required, sorting may not be. Heavy/large aggregations can be fairly CPU intensive. To improve performance, it would be great to have an option to disable sorting of results at both the global and shard-local level. Hash maps of results would be

Re: Post Filter On Nested Value

2015-01-09 Thread David Pilato
I think this should work: post_filter: { nested: { path: variants, filter:{ range: {price:{ gt: 20.0 }} } } } -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 9 janv. 2015 à 14:00, Dev Day d...@devday.com a écrit : Hi, I

Re: Post Filter On Nested Value

2015-01-09 Thread Dev Day
Hi David, Thank you for your reply help on this. Initially I did try this approach, but it didn't work. However, in order to provide you an example, I created a very simple doc. When I did this, I could see it worked. The issue I had with my code was that there was a price price at the root

Post Filter On Nested Value

2015-01-09 Thread Dev Day
Hi, I would like to run a post filter on a nested value, but cannot seem to get this working. This is my very simple document structure: {code:ABC, variants: [{sku:123, price: 15}], colour:blue} {code:DEF, variants: [{sku:456, price: 30}], colour:red} I have the following mapping {variants:

Re: Index corruption when upload large number of documents (4billion+)

2015-01-09 Thread Mano Ksoin
can you try with disable.allocaltion:true. this will tell es that dont allocate shards automatically. is the hard disk good or perhaps look at disk failure or bad track -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this