Re: Help with ES 1.x percolator query plz

2014-05-09 Thread Martijn v Groningen
I think the issue here is that the 'id' field is analyzed and your UUIDS are broken up into separate tokens. The standard analyzer is responsible for breaking up by '-'. If you use the analyze api you can see what happens with your uuids: curl -XGET

Re: Heisenbug with Percolator

2014-05-09 Thread Martijn v Groningen
Hi, Can you also share the document being percolated? I would expect the query to match if gasoline occurs in it and 'product' is equal to 'some_product'. You may have ran into a big regarding to the percolator and mappings: https://github.com/elasticsearch/elasticsearch/pull/5776 That has been

Re: Percolator on nested object shard failure

2014-05-09 Thread Martijn v Groningen
You would only get this error if the doc part in the request body is missing of the percolate request. Can you share the entire percolate request? On 6 May 2014 18:54, razafinr raz.r...@gmail.com wrote: I am using elasticsearch 1.1. Normally in this version, percolator on nested documents

Re: ANN: new elasticsearch discovery plugin - eskka

2014-05-09 Thread shikhar
Thanks for the feedback Otis! On Fri, May 9, 2014 at 12:33 AM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: At Sematext we use both ES and Akka (in SPM http://sematext.com/spm/), so this is interesting for me to see... Would it make sense to add a bit more to the README. things

Re: Mult-language searchable in one field

2014-05-09 Thread Martin
and I cant' use the _all field because of own scoring. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this

Re: Mult-language searchable in one field

2014-05-09 Thread Itamar Syn-Hershko
And then what analyzer you will use for that? It is doable, but I'd strongly suggest against it unless you know what you are doing: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_multi_field_2 -- Itamar Syn-Hershko http://code972.com |

Stability issues due to documention flaw ElasticSearch

2014-05-09 Thread Jasper Siero
We are investigating stability issues we encounter in our ES cluster, and feel that the ES documentation is lacking import information on this topic. The mailing list seems to fill the gap with the Document about recommended hardware specs sharding/index strategy

Re: Stability issues due to documention flaw ElasticSearch

2014-05-09 Thread Mark Walkom
Your link doesn't work :( However this sort of info is going to be in the ES (soon to be) book - http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/index.html Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web:

RE: Stability issues due to documention flaw ElasticSearch

2014-05-09 Thread Jasper Siero
The correct URL is: http://elasticsearch-users.115913.n3.nabble.com/Recommended-Hardware-Specs-amp-Sharding-Index-Strategy-td4039398.html The ES book would be really appreciated. Could you tell me when the release date is planned? Kind regards, Jasper Siero

Re: Stability issues due to documention flaw ElasticSearch

2014-05-09 Thread Mark Walkom
Well O'Reilly has it as March 2014, I'm guessing that is not the fully edited version - http://shop.oreilly.com/product/0636920028505.do Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 9 May 2014 21:46, Jasper Siero

Re: Stability issues due to documention flaw ElasticSearch

2014-05-09 Thread joergpra...@gmail.com
Because each Elasticsearch workload and hardware plus software is different, you should not expect a serious documentation can always give one-size-fits-all advisory. There are many rules of thumb and best practices, mostly based on individual experience on individual test cases. The foundation

Re: Stability issues due to documention flaw ElasticSearch

2014-05-09 Thread Paige Cook
Here is a post from the Elasticsearch Blog that gives some more context and status of the book - elasticsearch – the definitive guidehttp://www.elasticsearch.org/blog/elasticsearch-definitive-guide/ On Friday, May 9, 2014 8:03:27 AM UTC-4, Mark Walkom wrote: Well O'Reilly has it as March

Re: Stability issues due to documention flaw ElasticSearch

2014-05-09 Thread Mark Walkom
To add to this, you're still on 0.90.X and you should really upgrade as you will get the benefit of multiple fixes and updates. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 9 May 2014 22:19, joergpra...@gmail.com

Re: Issue with posting json data to elastic search via Flume

2014-05-09 Thread deepakas
I was on vacation . I am looking into it now. Thanks. -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Issue-with-posting-json-data-to-elastic-search-via-Flume-tp4054017p4055580.html Sent from the ElasticSearch Users mailing list archive at Nabble.com. -- You

Re: Tomcat and Elastic Search

2014-05-09 Thread benjelloun
Hello, can you describe steps to configure elasticSearch with tomcat6 or an exemple if you can ! thanks regards -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Tomcat-and-Elastic-Search-tp4018508p4055479.html Sent from the ElasticSearch Users mailing list

Realizing a prefix search across multiple tokens splitted by whitespace

2014-05-09 Thread Alexander Kluth
Although this is a topic which is discussed a lot on the internet I'm stuck with accomplishing one thing and I tried a lot unto this point. I want to have a prefix search on multiple tokens in a string (splitted by whitespaces). So, I've got the field Name with the content Hamburg Altona -

Is it possible to highlight the text with respect to no. of lines instead of no. of fragments

2014-05-09 Thread Anand kumar
Am having an index of huge content, from which I just want to have the highlighting of the specific text. The highlighted text might have appear as many as times, all I want to have two or more lines before and after the line of highlighted text, by which i can have a snippet of text with

Constant Score query Boost

2014-05-09 Thread Ilija Subasic
My understanding of Constant Score Query in elasticsearch is that boost factor would be assigned as score for every matching query. The documentationhttp://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-constant-score-query.html says: A query that wraps a filter or

ElasticSearch vs Solr integration with Tomcat6

2014-05-09 Thread anass benjelloun
Hello, I need to compare the both solutions ElasticSearch and Solr, then choose one of them to integrate it on my webapp, so i'm using tomcat6 server and i installed/Configured Solr.war in my webapp without any probleme then i search to integrate ElasticSearch with tomcat i didn't find enough

Re: Mult-language searchable in one field

2014-05-09 Thread Martin
Thank you - I thought may be there is another type like object-type which would provide it. Actually it should be a type like the array-type but with fields you could analyse separately. There would be different analysers for each language (especially because of stemming). -- You received

Re: Constant Score query Boost

2014-05-09 Thread 'Binh Ly' via elasticsearch
For the first one, you are probably experiencing score normalization happening. If you add explain: true to the query, it will tell you how it is being normalized. For the second one, if you want the function_score to totally override the Lucene score, you can replace the Lucene score with

Kibana 3 unable to connect to elasticsearch

2014-05-09 Thread Brian Wilkins
I am on RHEL 6. I can send messages from my Logstash shipper to Redis to Elasticsearch. I installed logstash via RPM on all my servers and I installed elasticsearch 1.0.3 via RPM. When I issue the command via curl to check my node status, I get two different versions. In Kibana 3, it tells me

Re: Mult-language searchable in one field

2014-05-09 Thread Martin
OK - I thought I could leave all my queries and add an additional filter if the item is available in that language. I'll watch your talk! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails

Re: Mult-language searchable in one field

2014-05-09 Thread joergpra...@gmail.com
You can index two fields into a third common field with index_name in combination with multi_field. This works like a custom _all field. Jörg On Fri, May 9, 2014 at 3:08 PM, Martin martin.konr...@echtzeiten.comwrote: OK - I thought I could leave all my queries and add an additional filter

Re: Kibana - Changing text color based on content

2014-05-09 Thread 'Binh Ly' via elasticsearch
You'll likely need to modify the panel code (module.js/module.html). In the Marvel panels, it's just some Javascript to change the css of the text element depending on what its value is. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To

Re: Constant Score query Boost

2014-05-09 Thread Ilija Subasic
Ahh, thanks. Thought as much about the normalization. Can it be turned off anywhere except in the mapping? Function score works with boost_mode=replace! On Friday, May 9, 2014 3:04:57 PM UTC+2, Binh Ly wrote: For the first one, you are probably experiencing score normalization happening. If

Kibana 3 Debian Packages.

2014-05-09 Thread David Bell
Hi there. I know that Kibana 3 is a very simple install already, but I am looking to Automate my server build completely. Are there any plans to create a Kibana repository to supply .deb packages. I am looking for this feature to simplify the install of a Logstash/Elasticsearch/Kibana server.

Re: Retrieve 6 products for top 3 users and each one has 2 with highest matching score

2014-05-09 Thread 'Binh Ly' via elasticsearch
Not sure if this would help, but you can first to a terms facet/aggregation on user_id, then you pull back the top 3 ids, say user5, user7, and user20. Then you run a second query using the _msearch API wherein you construct three independent search queries (one for each user) and you will get

Re: Query string operators seem to not be working correctly

2014-05-09 Thread 'Binh Ly' via elasticsearch
It's a feature of the query_string. What's happening is this query: sofa OR rugs AND red Actually means rugs and red must be there (always) to match. And if a document is a match (i.e. it contains both rugs and red) and it contains sofa also, then boost that document up some more ahead of the

Re: Mult-language searchable in one field

2014-05-09 Thread Martin
ah wow - that's cool. I just came across your http://jprante.github.io/lessons/2012/05/11/ISBN-search-with-Elasticsearch.html which explains it in detail. That's pretty close to what I want but I just found a new problem: There should be a highlighting of the found results - so I guess I'll

Re: Heisenbug with Percolator

2014-05-09 Thread Adam Georgiou
{ doc: { random field: [ \n\nMay 04--The 49ers employ an All-Pro linebacker whose college career appeared to foreshadow off-the-field trouble in the NFL. He was involved in an on-campus fight, suspended by his head coach and admitted to gasolining ],

Disk changes forced resync

2014-05-09 Thread Duncan Innes
Apologies if this is a silly question. I recently changed the disk layout on one of my ES nodes to put /var/lib/elasticsearch on it's own disk partition. Around 100Gb data was set to one side, new disk created, then rsync'd to it's new partition. As far as I can be certain, everything was the

Re: Is it possible to highlight the text with respect to no. of lines instead of no. of fragments

2014-05-09 Thread Nikolas Everett
On Fri, May 9, 2014 at 8:29 AM, Anand kumar anandv1...@gmail.com wrote: Am having an index of huge content, from which I just want to have the highlighting of the specific text. The highlighted text might have appear as many as times, all I want to have two or more lines before and after

Re: ElasticSearch vs Solr integration with Tomcat6

2014-05-09 Thread Hariharan Vadivelu
unlike SOLR , elasticsearch does not require a Java container, however you can always instantiate ES in embedded mode within your J2EE application. more details available here. http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html On Friday, May 9, 2014 7:43:56

Scoring a parent document search by a count of children matching part of the query?

2014-05-09 Thread Brian Jones
I have an index with parent documents ( Companies ), that have children ( Files ). Each Company can have hundreds of Files. Companies and Files both have many fields. The search I'm trying to perform is the Company that best matches based on it's own fields and the fields of it's children (

Re: ElasticSearch vs Solr integration with Tomcat6

2014-05-09 Thread Matt Weber
I would be best to manage elasticsearch outside of tomcat and use the java or rest api to communicate with ES from within your app. If you absolutely must run ES within tomcat, have a look at the wares transport[1]. [1] https://github.com/elasticsearch/elasticsearch-transport-wares Thanks, Matt

Re: Scoring a parent document search by a count of children matching part of the query?

2014-05-09 Thread Brian Jones
It seems like nesting the Files within the Company docs may be the only solution here. That is definitely an option. I had indexed the Files as children of Companies so that I could query the Files as a separate index ( which I also need to do ), but can maintain a separate index alltogether

Re: Query string operators seem to not be working correctly

2014-05-09 Thread Erich Lin
Thank you Binh. That validate API with explain is quite helpful. The feature seems a bit confusing because the API for query_string states that the precedence order of logical operators follow: AND first, then OR. Thus, when I see 'sofa OR rugs AND red', my brain would translate that into 1)

Re: Scoring a parent document search by a count of children matching part of the query?

2014-05-09 Thread Brian Jones
Are there any gotchas I should be aware of when creating a document that could contain thousands of pages of text ( a Company and thousands of nested Files ) in addition to dozens/hundreds of fields? On Friday, May 9, 2014 9:54:40 AM UTC-7, Brian Jones wrote: It seems like nesting the Files

Filtering nested aggregates

2014-05-09 Thread Ary Borenszweig
Hi, I have an index where I need to store medical test results. A test result can talk about many conditions and their results: for example, Tuberculosis = positive, Flu = negative. So I modeled my index like this: curl -XPUT http://localhost:9200/test_results/; -d' { mappings: {

Re: performance slowdown after upgrading from 0.90 to 1.1.1

2014-05-09 Thread Ivan Brusic
Which version of 0.90 did you upgrade from? Version 0.90.1 added store level throttle which can be a bottleneck if you have fast disks: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-store.html#store-throttling That is the only change I can think of that would

Re: performance slowdown after upgrading from 0.90 to 1.1.1

2014-05-09 Thread Kireet
We upgraded from 0.90.11, but we didn't set this option. Today we were able to reproduce the slowdown on a single node cluster. We also use some analysis plugins, especially the ICU plugin. We are going to try to check performance if we don't use those plugins to see if that's causing the

Faceting question

2014-05-09 Thread Ramdev Wudali
Hi : I have data that is like so : { Object1 : { text : This is text id: 123 }, Attribute2 : attribute_value }.. I want to facet (terms) on Object1.id, However, when the results are returned, I want the Text to be the value associated with the count. Is it

Re: Mapping created using Template does not work

2014-05-09 Thread Alexander Reelsen
Hey, can you just take some sample data and index it into elasticsearch manually and see if that works? --Alex On Thu, May 1, 2014 at 1:53 AM, Deepak Jha dkjhan...@gmail.com wrote: Hi, I have setup ELK stack and I am going by default index name, which is logstash-.MM.DD . Since this

Re: External datasets in ES

2014-05-09 Thread Alexander Reelsen
Hey, the tribe node (or federated search) is intended to join different clusters as one in order to execute operations against both. This is however only useful, if you do not have control over those two clusters (for example if they are managed by two different departments) or their data. In

Re: Query string operators seem to not be working correctly

2014-05-09 Thread Erich Lin
I believe I understand the problem now. 1) ES applies operators immediately to the left and right operand 2) ES does not virtually parenthesize groups after evaluating a higher precedence operator Thus, with a default operator of AND. Z B C OR D E F is interpreted as +Z + B C D +E +F My

Index Shard Number in Poor Performing Setup

2014-05-09 Thread webish
I'm seeing 40ms response times for queries using term and a custom ID field such as myId: 1234 I've inherited an Elastic Search database and application code. Currently there are two nodes and two main indexes. Index User is user profile data and Index News is event date in time series.

Re: Disk changes forced resync

2014-05-09 Thread Mark Walkom
In the past I have mounted the new partition under (eg) /mnt, then moved /var/lib/elasticsearch/* to it, then remounted the new partition to /var/lib/elasticsearch, and this has worked fine. However in your case the cluster may have rebalanced itself in the meantime, so when your node joined the

Re: Kibana 3 unable to connect to elasticsearch

2014-05-09 Thread Mark Walkom
It'll help if you can gist/pastebin your config for kibana. Also install a monitoring plugin like Marvel or ElasticHQ to give yourself better insight into your cluster. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com

Re: Mult-language searchable in one field

2014-05-09 Thread joergpra...@gmail.com
Elasticsearch has improved ithe index_name with the copy_to mechanism. I have set up a gist to demonstrate multilingual highlighting, maybe it is useful. https://gist.github.com/jprante/9f7523bb59ea654a3931 Jörg On Fri, May 9, 2014 at 4:19 PM, Martin martin.konr...@echtzeiten.comwrote: ah

Routing in Elasticsearch

2014-05-09 Thread Rajagopal Sathyamurthi
I have a question about how to best perform routing. We have three types of data that we are indexing -- entities, user_entity_relationships, and user_entity_analytics. This is mapped as a parent, child, grand child relationship. Entities being the parent - which can have multiple

Re: Kibana 3 unable to connect to elasticsearch

2014-05-09 Thread Brian Wilkins
Thanks for the tips. It turns it that it was a DNS issue. But I am going to install one of those tools. On Fri, May 9, 2014 at 5:22 PM, Mark Walkom ma...@campaignmonitor.com wrote: It'll help if you can gist/pastebin your config for kibana. Also install a monitoring plugin like Marvel or

Odd behavior with AND condition

2014-05-09 Thread mdj2
When I run the query (tags:(a)) in elasticsearch, I get 0 results. My query URL looks like: http://127.0.0.1:9200/haystack/_search?q=(tags%3A(%22a%22)) That is to be expected, since no objects have a tag set to a. Now when I change the condition, and add an AND, (org:(1) AND tags:(a)) , *I

2 nodes instead of one

2014-05-09 Thread Patrick Proniewski
Hello, I've noticed that my quite simple ES install (+ Logstash) has 2 nodes running, but I configured only one. Is this a normal behavior? And its health is yellow, as a consequence I guess of this second uninvited node. # egrep -v ^# /usr/local/etc/elasticsearch/elasticsearch.yml | sort -u

Re: 2 nodes instead of one

2014-05-09 Thread Ivan Brusic
It appears there is another Elasticsearch node on your network. Since you masked the value I cannot tell, but are transport_address and host the same for both nodes? If there is indeed another node on the network, you can either change the cluster name or disable multicast and use unicast. If you

Re: 2 nodes instead of one

2014-05-09 Thread Patrick Proniewski
I have only one server. host.domain.tld and PUB.LIC.IP.ADDR always represent the same values. It looks like the client node is started by Logstash, I don't think its useful, but I can't find a way to disable it. On 10 mai 2014, at 01:00, Ivan Brusic wrote: It appears there is another

Re: 2 nodes instead of one

2014-05-09 Thread Ivan Brusic
I am not a logstash expert, but did you perhaps enable an embedded version in the output section of your logstash config? -- Ivan On Fri, May 9, 2014 at 4:18 PM, Patrick Proniewski elasticsea...@patpro.net wrote: I have only one server. host.domain.tld and PUB.LIC.IP.ADDR always represent

Re: 2 nodes instead of one

2014-05-09 Thread Patrick Proniewski
No, it's even explicitly disabled since tonight. No difference. But may be it's a logstash bug (1.4.0) On 10 mai 2014, at 01:32, Ivan Brusic wrote: I am not a logstash expert, but did you perhaps enable an embedded version in the output section of your logstash config? -- Ivan On

Re: 2 nodes instead of one

2014-05-09 Thread Mark Walkom
What's your LS config, are you using http output? Also, install Marvel or ElasticHQ to give yourself a visual insight into your cluster. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 10 May 2014 10:02, Patrick

Re: Retrieve 6 products for top 3 users and each one has 2 with highest matching score

2014-05-09 Thread Yao
In multi search, you mean use 5, 7, 20 (user5, user7, and user20) to indicate user id for routing and then pick top 2 products for each of them? I use Play Framework and Scala, do you know how to embed the facet term results (user id) into the multi search? (As far as I know in ES Java APIs, it

Re: MoreLikeThis can't identify that 2 documents with exactly same attachments are duplicates

2014-05-09 Thread Zoran Jeremic
Thank you Alex. At the moment it works fine even with large documents, but I'll test if I can reach similar results with interesting terms. Best, Zoran On Thursday, 8 May 2014 02:02:24 UTC-7, Alex Ksikes wrote: On May 8, 2014 8:09 AM, Zoran Jeremic zoran@gmail.com javascript: wrote:

Find only those documents which do not contains id of some other nodes among results

2014-05-09 Thread Zoran Jeremic
Hi guys, I just solved one problem and got into the other, but I hope this would be a peace of cake for some of you with more experience with ES. I have documents that look like this: { _id:document_A_id, title:Document A, ... duplicates: [] } { _id:document_B_id, title:Document B,