Re: relevance in the range 0.0 to 1.0 ?

2014-11-06 Thread simonw
I think you should read this https://wiki.apache.org/lucene-java/ScoresAsPercentages it might help you to make a point. simon On Wednesday, November 5, 2014 8:42:59 PM UTC+1, Dustin Boswell wrote: Is there a way to score documents so that the relevance score has a fixed range, like from 0

Re: Difference in analyzer between 1.3.4 and 0.20.2

2014-11-06 Thread simonw
We fixed EdgeNGram tokenizer / filter in the 1.x series but don't ask me when exactly I think it was lucene 4.4 or so. Those offsets are now correct while they where broken before. not sure if this helps you to debug your problem On Thursday, November 6, 2014 1:31:22 PM UTC+1, Ben George wrote:

Re: Recovery failed : IllegalArgumentException[No enum constant org.apache.lucene.util.Version.4.3.1]

2014-07-28 Thread simonw
Hey, thanks for raising this. The exception is extremely helpful. This boils down to a bug / inconsistency in Lucene where some of the bugfix releases ie. 3.6.1, 4.3.1, 4.5.1 did specify the minor version in it's version constant where is shouldn't. I just opened [1] that fixes the issue. It's

Re: Recovery failed : IllegalArgumentException[No enum constant org.apache.lucene.util.Version.4.3.1]

2014-07-28 Thread simonw
FYI 1.3.1 has been release http://t.co/rCL9rAofta go ahead unless you wanna run on 1.3.1-SNAPSHOT thanks for raising this again! simon On Monday, July 28, 2014 5:17:46 PM UTC+2, Manikandan Pounraj wrote: Thanks Simon. We tested with latest build (1.3.1 SNAPSHOT). *All our old data

Re: limitation of 2,147,483,647 terms per segment index in Lucene

2014-07-02 Thread simonw
Peter, thanks so much for raising this. This looks aweful! I think we should move this into an issue on [1] (please feel free to create one) IMO we should aim to name the issue in a way to prevent this from happening altogether. Along the lines we should help you to recover but I don't know

Re: corruptindexexception warn message unclear

2014-06-30 Thread simonw
hey, thanks for raising this, can you gimme more infos ie. which version you are using and if that happens only on one shard or on all shards in your system? It could just be what it says, and index corruption maybe due to HW failure but there could be other reasons simon On Friday, June

Re: inexplicable behavior using 'terms' filter

2014-04-25 Thread simonw
to be able to eliminate all the elements that are not ['apple', 'banana']. About the results I'm building an example, and soon as it will be running I'll send it here, hopefully by the end of the day. thanks again! Luca On Thu, Apr 24, 2014 at 6:42 PM, simonw simon.w

Re: inexplicable behavior using 'terms' filter

2014-04-24 Thread simonw
luca, what do the values in 'importantId` look like? Are they identical to the values you pass in? The field is not analyzed though. It would also be helpful if you could show the results that you would expect so we can see what values they have. simon On Thursday, April 24, 2014 5:53:42 PM

Re: Lucene index corruption on nodes restart

2014-04-02 Thread simonw
hey, is it possible to look at this index / shard? do you still have it / can you safe it for further investigations? You can ping me directly at simon AT elasticsearch DOT com On Wednesday, April 2, 2014 11:23:38 AM UTC+2, Paweł Chabierski wrote: Few days ago we found we've got that same

Re: Lucene index corruption on nodes restart

2014-03-27 Thread simonw
Regarding the corruption - that is a lucene problem that we are working on to address. The corruption might have happened a while ago already but you are only happen to see it now. There is no integrity checks in lucene yet so if there is a corrputed index due to HW failure you will also

Re: IndexOutOfBoundsException at IndexShardRoutingTable class

2014-03-27 Thread simonw
I have to thank you for bringing it up! :) On Wednesday, March 26, 2014 9:40:26 PM UTC+1, Shinsuke Sugaya wrote: Thank you for the investigation and filing it. Regards, shinsuke On Thursday, March 27, 2014 12:55:56 AM UTC+9, simonw wrote: I opened an issue for this: https://github.com

Re: [BUG?] match_phrase_prefix failing on v1.1.0, working on 1.0.2

2014-03-26 Thread simonw
It's actually unrelated to this issue mentioned above. It got broken due to https://github.com/elasticsearch/elasticsearch/pull/5005 and there is an issue open here: https://github.com/elasticsearch/elasticsearch/issues/5551 a fix is right here:

Re: IndexOutOfBoundsException at IndexShardRoutingTable class

2014-03-26 Thread simonw
that is actually a bug IMO - Math.abs() can return -1 if it hits Integer.MIN_VALUE this code is just broken - can you open an issue! On Wednesday, March 26, 2014 11:43:20 AM UTC+1, Kevin Wang wrote: pickIndex() will return the absolute value of the count, so it won't return a negative

Re: IndexOutOfBoundsException at IndexShardRoutingTable class

2014-03-26 Thread simonw
I opened an issue for this: https://github.com/elasticsearch/elasticsearch/issues/5559 On Wednesday, March 26, 2014 5:53:15 AM UTC+1, Shinsuke Sugaya wrote: Hi I encountered the following problem: Caused by: java.lang.IndexOutOfBoundsException: index (-2) must not be negative at

Re: trivial example of keyword_repeat?

2014-03-13 Thread simonw
the repeat filter only applies to terms that actually get stemmed. ie if you have goes it will be stemmed to go but with the repeat filter it will also emit goes in addition to go makes sense? simon On Thursday, March 13, 2014 12:38:00 AM UTC+1, Nikita Tovstoles wrote: Could someone please

Re: ElasticSearch server lock up

2014-02-27 Thread simonw
We have several fixes relate to this in 0.90.12 and 1.0.1 - can you guys try and upgrade? would be awesome if you could report back! simon On Thursday, January 23, 2014 1:47:31 AM UTC+1, JBartak wrote: We had an interesting problem today while running a 3 data node ElasticSearch 90.7

Re: ElasticSearch server lock up

2014-02-11 Thread simonw
do not have log message from the beginning of this event. On Thursday, February 6, 2014 3:18:14 PM UTC-5, simonw wrote: Thanks so much for the infos here! That is really helpful! Just to keep you updated, I committed a fix for the infinite loop to lucene 5 min ago ( https

Re: ElasticSearch server lock up

2014-02-11 Thread simonw
started occurring around 4:16am EST and continued until 4:45am EST. We noticed issues started with a bulk index request (not sure if that helps at all). On Tuesday, February 11, 2014 4:21:12 AM UTC-5, simonw wrote: Jay, first of all good that this prevents the server from going

Re: ElasticSearch server lock up

2014-02-06 Thread simonw
Hey folks, we are trying to get to the bottom of this and I'd want to have some more infos here. Can you provide us with more insight in what you are doing with ES. What I'd be interested in is: * are you using delete by query * are you running parent child queries * are you seeing any

Re: ElasticSearch server lock up

2014-02-06 Thread simonw
during normal runtime of our application * We have seen the issue in 0.20.2 (JDK6) and 0.90.8 (JDK7u45) * No not forcefully flushing On Thursday, February 6, 2014 2:47:38 PM UTC-5, simonw wrote: Hey folks, we are trying to get to the bottom of this and I'd want to have some more infos here

Re: Score depending on position in the term on the field

2014-01-24 Thread simonw
maybe this is interesting in your situation? http://www.elasticsearch.org/blog/you-complete-me/ simon On Thursday, January 23, 2014 4:15:05 PM UTC+1, Nikolay Chankov wrote: Well, it's really strange, that position is not encountered, since with many results and especially data with lot of

Re: Newer elasticsearch instance cannot join the cluster

2014-01-13 Thread simonw
All these version are backwards compatible. You can safely upgrade a lucene index from 4.3.1 to 4.5.1 or 4.6.0. You can't downgrade so they are not forward compatible though! simon On Monday, January 13, 2014 1:40:47 AM UTC+1, Jason Wee wrote: Hi, for es version 0.90.2 using lucene 4.3.1