[ANN] Elasticsearch Google Compute Engine cloud plugin 2.1.1 released

Heya, We are pleased to announce the release of the Elasticsearch Google Compute Engine cloud plugin, version 2.1.1. The Google Compute Engine (GCE) Cloud plugin allows to use GCE API for the unicast discovery mechanism.. https://github.com/elasticsearch/elasticsearch-cloud-gce/ Release Not

[ANN] Elasticsearch Google Compute Engine cloud plugin 2.0.1 released

Heya, We are pleased to announce the release of the Elasticsearch Google Compute Engine cloud plugin, version 2.0.1. The Google Compute Engine (GCE) Cloud plugin allows to use GCE API for the unicast discovery mechanism.. https://github.com/elasticsearch/elasticsearch-cloud-gce/ Release Not

[ANN] Elasticsearch Google Compute Engine cloud plugin 1.3.0 released

Heya, We are pleased to announce the release of the Elasticsearch Google Compute Engine cloud plugin, version 1.3.0. The Google Compute Engine (GCE) Cloud plugin allows to use GCE API for the unicast discovery mechanism.. https://github.com/elasticsearch/elasticsearch-cloud-gce/ Release Not

Need to get Elastic Search current highest id

Hi All, I am new to elastic search. I am working on river jdbc mysql concepts I have a scenario like I need to get highest value for field "id" that is inserted in to elastic search.please suggest me how to build this query to get highest value for id inserted. Thanks & Regards, phani. --

Cluster configuration

hello, i am trying to implement elasticsearch cluster. i have 2 machines with 2 nodes each. i have following configuration in yml file. i have given unique node name on each node, all of them are master and data nodes. cluster.name: elasticsearch node.master: true node.data: true discovery.zen

Re: Analyzing URLs for regexp queries

Luiz, thanks for responding! I had forgotten to mention I tried not_analyzed as well. The analyzer it turns out wasn't my problem. I had 2 problems. First, the ES/Lucene regexp query/filter doesn't support "\d" for indicating digits. So I had to replace them with the [0-9] character class. Once I

Re: how to control the returned record number in filter?

hi Adrien, Thanks for your response! Actually, the basic requirement is at https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/elasticsearch/kiQ0FGFKUkA I asked some questions all about this requirement, now I know I can get my expected result using multi search, as I s

searching most recent objects

I am using the javascript API and want to do a search and have it search through the most recent objects, IE I call a search with size 100, I want to have the most recent 100 objects returned to me, how would I go about doing that? I tried using sort, but it seems that it just sorts the results

*same* query return different result

I am using version 1.0.0.RC2 for the following two queries: one is using match_all, the other is using term filter "query": { "match_all": {} }, "aggregations": { "by_campaign": { "terms": { "field": "campaign_name" } } } ***

Re: Analyzing URLs for regexp queries

Hi Matt, If you mark your field as not_analyzed: { "mappings": { "type1": { "properties": { "url": { "type": "string", "index": "not_analyzed" } } } } } You could use a regexp q

Re: Scroll and Scan

Thanks Brian. I am using the scroll_id from the SearchResponse returned with each scroll request for the subsequent scroll request. And yes, I am repeating the scan until there are no more results. However, I was expecting to get n*# of shards returned with each request, but I seem to be gettin

[Hadoop] writing ES string array from Pig using elasticsearch-hadoop plugin

I am having problems while trying to write an ES string array from my Pig script. The current ES-hadoop documentation ( http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/pig.html) states that a pig Bag maps to a ES Array. It sounds like you should be able to stick all of the i

Analyzing URLs for regexp queries

I have a field in my documents that consists of a URL. {... "url":"http://example.com/2014/04/15/foo-bar-baz/"; ...} I would like to use a regexp query/filter to find documents in my index with urls matching a regex pattern. For example: "http://example\.com/\d{4}/\d{2}/\d{2}/([^/]+)/$" I'm a

Re: Is it possible to add a customized merging strategy to alleviate split-brain impact?

Anyone, please? On Monday, March 31, 2014 11:11:56 AM UTC-7, Jing Liu wrote: > > Hi ES team, > > When split-brain occurs, I found following behaviors on ES during the > merge between A and B (i.e., a group of nodes with master A or B): > Assume we don't know when the split-brain happens and both

Re: garbage collection log messages, [monitor.jvm ... duration [2.2m]

Thanks for the explanation, that really helps. Does that mean that on a virtual host with 64GB memory it might make sense to make two virtual servers each running a node? I had expected that multiple nodes on a single host would not help, but I guess if the VM is the limitation it might? I have a

Array scoring: max instead of sum

Hi, My documents have an "aliases" field which is an array of string. When I query that field, ES scores each matching alias and uses the sum of these scores to calculate the document's score. However, I would like to use the max of these scores. I don't care if a document has many aliases ma

Re: Scroll and Scan

Robbie, Are you repeating the scan until there are no more hits returned? I've never bothered to check the repeatability of the individual "chunks" and only notice that the overall total count is as expected. Also note the following from the guide: The scroll request also returns a *new _scrol

Nested Document sorting and filtering

Hi I have a document structure as follows productId, productName, [ {tag1, tag_name, price, tag_category}, {tag2, tag_name, price, tag_category}, ...] where tags is a nested document within a product. Each product is a unique item, but different merchants can sell the same product (called tags

Re: Scroll and Scan

One another issue that I am seeing is that sometimes the scroll api returns a different number of results than what I expect. For instance if I have Index A with 5 shards, Index B with 5 shards, and both indices have 10K records which match my filter, if my scroll size is 10 per shard, I somet

Re: Json for Indexing Data in Elastic Search using C#.Net

Try starting here: https://www.nuget.org/packages/NEST -- Itamar Syn-Hershko http://code972.com | @synhershko Freelance Developer & Consultant Author of RavenDB in Action On Tue, Apr 15, 2014 at 9:30 AM, Amit Tripathi wrote: >

Distributed Tracing logging and Integrating with Kibana

I have implemented a Distributed Transaction Logging library with Tree like Structure as mention in Google Dapper( http://research.google.com/pubs/pub36356.html) and eBay CAL Transaction Logging Framework( http://devopsdotcom.files.wordpress.com/2012/11/screen-shot-2012-11-11-at-10-06-39-am.png)

Re: Querying with Completion Suggesters with Java API

The previous example is missing the first startObject/endObject: new PutMappingRequestBuilder(newClient .admin().indices()) .setIndices(INDEX_NAME) .setType(INDEX_TYPE) .setSource(jsonBuilder().startObject().startObject(INDEX_TYPE) .startObject("properties") .startObject("suggest") .field("type",

Re: How to count filtered aggregations on nested types at parent level?

This sounds exactly like what I need! I guess I can live with my workaround until reverse_nested becomes available. Thanks, Dan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it,

Re: ES cluster unable to assing new shards

Hi Mark, I forgot to write it again in this mail, but in the gist I pasted the full logs when the ES cluster created the new indices until I tried to restart the current active master. # head es_cluster.log [2014-04-14 02:00:01,504][INFO ][cluster.metadata ] [es@log01] [logstash-2014.0

Snapshot to S3

I'm trying to setup S3 backup with Elasticsearch 1.1.0 new snapshot feature. For that I installed the cloud-aws plugin https://github.com/elasticsearch/elasticsearch-cloud-aws#s3-repository I followed this instruction by adding my bucket like that : curl -XPUT 'http://localhost:9200/_snapshot/m

Re: search pattern with a forward slash

Make it a phrase query: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#_phrase or with the query_string: { "bool": { "must": { "query_string": { "query": "(symbol:\"AUD/USD\")" } } }

Re: ES cluster unable to assing new shards

Check your ES logs, there may be something there. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 15 April 2014 22:20, Andreas Paul wrote: > Hello there, > > on Monday morning our ES cluster cluster switched to red

search pattern with a forward slash

Hi, I have a set of data setting in elasticsearch, like {"symbol":"AUD/USD"},{"symbol":"HKD/USD"},{"symbol":"AUD/EUR"}... I would like to search for "AUD/USD". So the search query string will be "{ "bool":{ "must":{ "query_string":{ "query":"(symbol:AUD/USD)" } } } }". But the result I got lo

Re: Querying with Completion Suggesters with Java API

You're missing the index 'type' and 'properties' objects from the index mapping. Change it to: new PutMappingRequestBuilder(newClient .admin().indices()) .setIndices(INDEX_NAME) .setType(INDEX_TYPE) .setSource(jsonBuilder().startObject(INDEX_TYPE) .startObject("properties") .startObject("suggest"

Re: garbage collection log messages, [monitor.jvm ... duration [2.2m]

The advisory of "a lot of heap" means, give as much heap as the JVM is able to process efficiently. There is an upper limit due to JVM engineering state of today. You will not find JVMs that can efficiently manage heaps >32G (except rare expensive commercial JVM products). By efficient I mean GC st

Calculating ArcDistance of a geopoint field within an object field

I am trying to obtain geodistance values for records in cases where geodistance is not the primary purpose of the search (i.e. do not want it to affect score/sort not included in a query/filter), but where that information might be useful anyway. As outlined in these example cases https://gist.

Re: garbage collection log messages, [monitor.jvm ... duration [2.2m]

On Tue, Apr 15, 2014 at 9:42 AM, Wouter van Atteveldt < wou...@vanatteveldt.com> wrote: > > > > On Tue, Apr 15, 2014 at 2:00 PM, joergpra...@gmail.com < > joergpra...@gmail.com> wrote: > >> This is not Elasticsearch related. If you use a 40g heap of such extreme >> size, you must expect that garba

Re: boosting for a number near X

Hi Guy, sounds like you are looking for the function score query: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html And there maybe especially for the decay functions: "Decay functions score a document with a function that decays dependi

Re: garbage collection log messages, [monitor.jvm ... duration [2.2m]

On Tue, Apr 15, 2014 at 2:00 PM, joergpra...@gmail.com < joergpra...@gmail.com> wrote: > This is not Elasticsearch related. If you use a 40g heap of such extreme > size, you must expect that garbage collection must run for minutes, on > every JVM I know. > > Right, but it is actually advised to gi

Re: Querying with Completion Suggesters with Java API

Hey Dan! Thanks for the code snippet, it's actually helped me narrow down my problem considerably. I'm getting the error; field 'suggest' is not a completion suggest field. To my knowledge, you define it as a completion suggest field with the type in the mapping. .field("suggest").startObject()

boosting for a number near X

Hi, I have documents (accommodations) stored with a numeric attribute - "number of beds" I'd like to be able to boost by a certain amount if that number is close to a search query. e.g. prefer documents with a number of beds close to what is being searched for. This will be added to the rest

Re: getting the max theoretical score for a search query

Many thanks Adrien On Friday, April 11, 2014 1:57:34 AM UTC+2, Adrien Grand wrote: > > Hi, > > Although it is generally not advised to return scores as percentages[1], > in this particular case it would work given that you are only querying > structured attributes and don't care about term frequ

ES cluster unable to assing new shards

Hello there, on Monday morning our ES cluster cluster switched to red, because he didn't assign the new created indices to any ES node, see attached picture. I tried manually allocating these unassigned shards to a node, but it only returned the following error: # curl -XPOST $(hostname -f):

Re: garbage collection log messages, [monitor.jvm ... duration [2.2m]

This is not Elasticsearch related. If you use a 40g heap of such extreme size, you must expect that garbage collection must run for minutes, on every JVM I know. Jörg On Tue, Apr 15, 2014 at 11:35 AM, Wouter van Atteveldt < vanatteve...@gmail.com> wrote: > Dear Binh Ly, > > Thanks for your repl

Local testing not working

Hi I am just starting up with elastic search and I have creaed a local builder like this public class EmbeddedElasticSearchServer { private static final String DEFAULT_DATA_DIRECTORY = "data/elasticsearch-data"; private final Node node; private final String dataDirectory;

Re: Percolator - Match all words

This works well! Thanks a lot! On Tuesday, April 15, 2014 5:05:45 AM UTC-4, David Pilato wrote: > > May be this? > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#_phrase > > -- > *David Pilato* | *Technical Advocate* | *Elasticsearch.com* > @da

Re: [Hadoop] storing data in ES using pig script

Glad to hear it but know that the latest release is 1.3.0 M3. Simply check the official project page [1] and you get all the info [2], including the download setup from Maven, for both stable and dev/snapshot releases [3] [1] http://www.elasticsearch.org/overview/hadoop/ [2] http://www.elastics

Re: cardinality not giving expected results.

Thank you for this report, this indeed looks like a bug. Do you have the same issue if you use the cardinality aggregation as a top-level aggregation (the way it is executed is a bit different in that case so I am thinking the bug might only happen when used as a sub aggregation). On Tue, Apr 15,

Re: How to count filtered aggregations on nested types at parent level?

It is not possible to count parent documents yet, but this will hopefully be available in Elasticsearch 1.2.0 via the `reverse_nested` aggregation[1], that would be able to translate back nested doc IDs to parent doc IDs. [1] https://github.com/elasticsearch/elasticsearch/issues/5485 On Tue, Apr

Re: How to run example of context suggester in elasticsearch doc?

Hi, The context suggester will be available in Elasticsearch 1.2.0, it has not been released yet. (see note at the top of the documentation) On Tue, Apr 15, 2014 at 8:14 AM, wrote: > I am learning the elasticsearch, and I want to run the example of context > suggester following the doc: > http

Re: elasticsearch: How to set custom record count based on a type in search result

Hi, It is not possible to do it in a single query. I think the easiest way to do that would be to issue several queries and merge results on client side. On Tue, Apr 15, 2014 at 8:07 AM, wrote: > > Anyone can give any suggests? thanks! > > -- > You received this message because you are subscri

Re: Can I use the filter in multi search?

This will work fine by default: filters are cached, so the first query will load the set of matching documents into the cache and the other queries will be able to reuse this cache entry. On Tue, Apr 15, 2014 at 10:21 AM, wrote: > I have a requirement need to use multi search, but the search h

Re: how to control the returned record number in filter?

Can you tell us more about what you are trying to achieve? On Tue, Apr 15, 2014 at 10:23 AM, wrote: > In ES, we can use "from" and "size" to control the returned record number > in query, but can I also control them in filter? > the limit filter is worked on every shard, so I think it's not wor

Re: how to merge the result after multi search, then I can sort them like sorting in queryDSL?

Hi, All that the multi-search can do is executing several search requests in a single HTTP request. If you want to have merged results for both topics, you either need to submit a single query that matches both topics (but there is no guarantee that you will have 4 of them in the business topic an

Re: java 8, elasticsearch, and MVEL

Is there an open issue so that I can watch the progress for this bug? I cannot find any issue for this on GitHub. Am 07.04.2014 01:12, schrieb Shay Banon: We will report back with findings and progress. -- You received this message because you are subscribed to the Google Groups "elasticsea

Re: garbage collection log messages, [monitor.jvm ... duration [2.2m]

Dear Binh Ly, Thanks for your reply and sorry for not responding earlier. We've moved over our elasticsearch to SSD and I had hoped that that might help with the performance issues, but no luck. It seems that whenever elastic is freshly started it performs pretty well, but after a couple days

Re: TML - Text Mining Library for LSA (Latent Semantic Analysis) implemetation

Hello Naveent , Can you please give more ideas on what you are looking for. As far as i know , only the carrot2 plugin has scratched that surface for ES - https://github.com/carrot2/elasticsearch-carrot2 Thanks Vineeth On Tue, Apr 15, 2014 at 11:28 AM, Navneet Mathpal < navneetmathp

Re: [Hadoop] storing data in ES using pig script

Hello , I used "elasticsearch-hadoop-1.3.0.M2" and it given me Failed Jobs: JobIdAliasFeatureMessageOutputs job_201404142111_0008weblog_count,weblog_group,weblogs GROUP_BY,COMBINERMessage: Job failed! Error - # of failed Reduce Tasks exceeded allowed limit. FailedCoun

Re: Percolator - Match all words

May be this?  http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#_phrase --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 15 avril 2014 à 06:15:21, Louis-René Haché (talkt...@lrhache.com) a écrit: Hi As show

Lingo3g Attribute to retrieve labels name with suppressing multiple keyword

Hi ES users, Is there any attribute in lingo3g to suppress the label name returned by the ES with respect to multiple keyword. For ex. If my cluster query returns label name as : 1) India development , india , hello india 2) mobile samsung , motorola g , micromax canvas So is there any attribute

Re: [Hadoop] java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Excellent. The fix will be part of RC1 which is not too far off. Cheers, On 4/15/14 11:30 AM, Shaun Campbell wrote: Thanks Costin, that has fixed it! :) -Original Message- From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of Costin Leau Sent: 15 A

RE: [Hadoop] java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Thanks Costin, that has fixed it! :) -Original Message- From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of Costin Leau Sent: 15 April 2014 08:58 To: elasticsearch@googlegroups.com Subject: Re: [Hadoop] java.lang.IncompatibleClassChangeError: Found i

how to control the returned record number in filter?

In ES, we can use "from" and "size" to control the returned record number in query, but can I also control them in filter? the limit filter is worked on every shard, so I think it's not work for me. thanks johnson -- You received this message because you are subscribed to the Google Groups "e

Can I use the filter in multi search?

I have a requirement need to use multi search, but the search has same conditions except one, so I hope I can search them using the same conditions and using filter on different condition, so I think the performance is better, does anyone have any suggest? thanks Johnson -- You received thi

how to merge the result after multi search, then I can sort them like sorting in queryDSL?

I have a multi search like: {"index" : "news"} {"query":{ "bool":{ "must":[{ "more_like_this_field" : { "content" :{"like_text" : "James", "min_term_freq":1, "max_query_terms" : 12}} }, { "match":{"topic":"Business"} }] }}, "from" : 0, "size" : 4} {"index" : "news"} {"query":{ "bool":{ "mu

how to merge the result after multi search, then I can sort them like sorting in queryDSL?

I have a multi search like: {"index" : "news"} {"query":{ "bool":{ "must":[{ "more_like_this_field" : { "content" :{"like_text" : "James", "min_term_freq":1, "max_query_terms" : 12}} }, { "match":{"topic":"Business"} }] }}, "from" : 0, "size" : 4} {"index" : "news"} {"query":{ "bool":{ "mu

Re: [Hadoop] java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Hi, You bumped into a bug that slipped in M3. This has been fixed in master - can you please try out the SNAPSHOTs [1]? Thanks, [1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/install.html#download-dev On 4/15/14 10:52 AM, Shaun Campbell wrote: Hi there, I'm having a

[Hadoop] java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Hi there, I'm having a bit of trouble using the Maven published version of elasticsearch-hadoop 1.3.0M3. According to the docs, it should be compatible with both Hadoop 1.x and 2.x, although I'm seeing an error indicating otherwise. I downloaded the sources, and modified the build to compile a

Re: Regexp "intersection" matching nothing

Ok, sorry for the noise, regexp filter simply doesn't seem to like analyzed fields... -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr..