Why release 3.0?

2009-11-16 Thread Erick Erickson
One of my specialties is asking obvious questions just to see if everyone's assumptions are aligned. So with the discussion about branching 3.0 I have to ask Is there going to be any 3.0 release intended for *production*?. And if not, would we save a lot of work by just not worrying about

Re: Why release 3.0?

2009-11-16 Thread Jake Mannix
Don't users need to upgrade to 3.0 because 3.1 won't be necessarily able to read your 2.4 index file formats? I suppose if you've already upgraded to 2.9, then all is well because 2.9 is the same format as 3.0, but we can't assume all users upgraded from 2.4 to 2.9. If you've done that already,

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
will only go into 3.x. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de _ From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Monday, November 16, 2009 7:10 PM To: java-dev@lucene.apache.org Subject: Why release 3.0? One

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
release 3.0? Don't users need to upgrade to 3.0 because 3.1 won't be necessarily able to read your 2.4 index file formats? I suppose if you've already upgraded to 2.9, then all is well because 2.9 is the same format as 3.0, but we can't assume all users upgraded from 2.4 to 2.9. If you've

Re: Why release 3.0?

2009-11-16 Thread Jake Mannix
*Subject:* Re: Why release 3.0? Don't users need to upgrade to 3.0 because 3.1 won't be necessarily able to read your 2.4 index file formats? I suppose if you've already upgraded to 2.9, then all is well because 2.9 is the same format as 3.0, but we can't assume all users upgraded from

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -- *From:* Jake Mannix [mailto:jake.man...@gmail.com] *Sent:* Monday, November 16, 2009 7:15 PM *To:* java-dev@lucene.apache.org *Subject:* Re: Why release 3.0

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
...@thetaphi.de _ From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:09 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? uwe, on topic please read my comment on LUCENE-1689, because unicode version was bumped in jdk 1.5, i believe this index

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
://www.thetaphi.de eMail: u...@thetaphi.de -- *From:* Robert Muir [mailto:rcm...@gmail.com] *Sent:* Monday, November 16, 2009 8:09 PM *To:* java-dev@lucene.apache.org *Subject:* Re: Why release 3.0? uwe, on topic please read my comment on LUCENE-1689, because

RE: Why release 3.0?

2009-11-16 Thread Steven A Rowe
-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:09 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0

Re: Why release 3.0?

2009-11-16 Thread Mark Miller
*To:* java-dev@lucene.apache.org mailto:java-dev@lucene.apache.org *Subject:* Re: Why release 3.0? Don't users need to upgrade to 3.0 because 3.1 won't be necessarily able to read your 2.4 index file formats? I suppose if you've already upgraded to 2.9

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
Bremen http://www.thetaphi.de eMail: u...@thetaphi.de From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:09 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
://www.thetaphi.de eMail: u...@thetaphi.de _ From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:09 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? uwe, on topic please read my comment on LUCENE-1689, because unicode version was bumped in jdk 1.5, i

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:37 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? right, its nothing to do with lucene, instead due to property changes, etc. i just think we should inform users on java 1.4/2.9 that if they upgrade to java 1.5

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
] Sent: Monday, November 16, 2009 7:15 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? Don't users need to upgrade to 3.0 because 3.1 won't be necessarily able to read your 2.4 index file formats? I suppose if you've already upgraded

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
http://www.thetaphi.de eMail: u...@thetaphi.de _ From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:45 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? right, my point is its true its nothing to do with Lucene at all, really. but the reality

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -- *From:* Robert Muir [mailto:rcm...@gmail.com] *Sent:* Monday, November 16, 2009 8:45 PM *To:* java-dev@lucene.apache.org *Subject:* Re: Why release 3.0? right, my point is its true its nothing to do

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
-- *From:* Robert Muir [mailto:rcm...@gmail.com] *Sent:* Monday, November 16, 2009 8:45 PM *To:* java-dev@lucene.apache.org *Subject:* Re: Why release 3.0? right, my point is its true its nothing to do with Lucene at all, really. but the reality is we

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:52 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? Uwe, thats probably a good solution I think. just as long as we document somewhere, I think there is some warning verbage in StandardTokenizer already about

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
-dev@lucene.apache.org Subject: Re: Why release 3.0? btw, so heres a great example. you are backwards broken regardless of JVM for StandardTokenizer, because we used 1.4 JRE to run jflex in 2.9, but 1.5 in 3.0, right? On Mon, Nov 16, 2009 at 2:51 PM, Robert Muir rcm...@gmail.com wrote: Uwe

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
:* Robert Muir [mailto:rcm...@gmail.com] *Sent:* Monday, November 16, 2009 8:53 PM *To:* java-dev@lucene.apache.org *Subject:* Re: Why release 3.0? btw, so heres a great example. you are backwards broken regardless of JVM for StandardTokenizer, because we used 1.4 JRE to run jflex in 2.9

Re: Why release 3.0?

2009-11-16 Thread Erick Erickson
...@gmail.com] *Sent:* Monday, November 16, 2009 7:10 PM *To:* java-dev@lucene.apache.org *Subject:* Why release 3.0? One of my specialties is asking obvious questions just to see if everyone's assumptions are aligned. So with the discussion about branching 3.0 I have to ask Is there going

Re: Why release 3.0?

2009-11-16 Thread Erick Erickson
-- *From:* Erick Erickson [mailto:erickerick...@gmail.com] *Sent:* Monday, November 16, 2009 7:10 PM *To:* java-dev@lucene.apache.org *Subject:* Why release 3.0? One of my specialties is asking obvious questions just to see if everyone's assumptions are aligned. So

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
Java 1.4 can be seen by trying out. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de _ From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 9:06 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 8:09 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? uwe, on topic please read my comment on LUCENE-1689, because unicode version was bumped

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
...@gmail.com] Sent: Monday, November 16, 2009 9:13 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? Oops, stupid mouse made me send a blank message. Ok, I withdraw the question since there *are* good reasons to put 3.0 in a prod environment G. It's also an easier thing to say new

Re: Why release 3.0?

2009-11-16 Thread Mark Miller
:* Re: Why release 3.0? btw, so heres a great example. you are backwards broken regardless of JVM for StandardTokenizer, because we used 1.4 JRE to run jflex in 2.9, but 1.5 in 3.0, right? On Mon, Nov 16, 2009 at 2:51 PM, Robert Muir rcm...@gmail.com mailto:rcm

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de _ From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, November 16, 2009 9:15 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? Steven, I think we can be almost sure of no latin-1

Re: Why release 3.0?

2009-11-16 Thread Mark Miller
Muir [mailto:rcm...@gmail.com] *Sent:* Monday, November 16, 2009 8:52 PM *To:* java-dev@lucene.apache.org *Subject:* Re: Why release 3.0? Uwe, thats probably a good solution I think. just as long as we document somewhere, I think there is some warning verbage in StandardTokenizer already

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Monday, November 16, 2009 9:21 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? Good point - and that likely means

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
@lucene.apache.org *Subject:* Re: Why release 3.0? btw, so heres a great example. you are backwards broken regardless of JVM for StandardTokenizer, because we used 1.4 JRE to run jflex in 2.9, but 1.5 in 3.0, right? On Mon, Nov 16, 2009 at 2:51 PM, Robert Muir rcm...@gmail.com

Re: Why release 3.0?

2009-11-16 Thread Mark Miller
:21 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? Good point - and that likely means the current warning is not working - what can we do to improve it? Perhaps a new text file called jflexregen or something, and it specifically says you must use java 1.5? Uwe Schindler

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
://www.pangaea.de/ E-mail: uschind...@pangaea.de -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Monday, November 16, 2009 9:30 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? And what happens when someone regenerates it with 1.6 without knowing

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? And what happens when someone regenerates it with 1.6 without knowing? Uwe Schindler wrote: I check this by generating the file with 1.4 and 1.5. The 1.4 version will not change anymore, so we just leave the java file

Re: Why release 3.0?

2009-11-16 Thread Mark Miller
[mailto:markrmil...@gmail.com mailto:markrmil...@gmail.com] Sent: Monday, November 16, 2009 9:30 PM To: java-dev@lucene.apache.org mailto:java-dev@lucene.apache.org Subject: Re: Why release 3.0? And what happens when someone regenerates it with 1.6 without knowing

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
9:45 PM To: java-dev@lucene.apache.org Subject: Re: Why release 3.0? I still reccomend we add a file then HowToRegenJflex.txt or something - that specifically says to use 1.5 or 1.6. I don't changing the current notice/warning is visible enough to ensure someone doesn't break this. Robert

RE: Why release 3.0?

2009-11-16 Thread Uwe Schindler
to java-dev users. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Monday, November 16, 2009 9:59 PM To: java-dev@lucene.apache.org Subject: RE: Why release 3.0

Re: Why release 3.0?

2009-11-16 Thread DM Smith
On Nov 16, 2009, at 6:43 PM, Robert Muir wrote: DM, in this case I'm not referring to surrogates, etc, but instead the idea that properties for an existing character can change (the soft hyphen and arabic ayah were two examples), also new characters are introduced. these will affect what

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
Is core lucene really affected by the change? Or is it only contrib? I mean, if we couldn't create an index using core with surrogate pairs and other Unicode 4.0 stuff (though I'm not clear on the changes), how can it change reading/searching the index? Sure, especially core analyzers like

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
right, the only way you could really contain it would be to do something like that. I just think we should make users aware of this, thats all. and I think it sucks they might have to reindex twice with the current status of things (we did not complete unicode 4 support in lucene 3.0) which is

Re: Why release 3.0?

2009-11-16 Thread DM Smith
On Nov 16, 2009, at 7:53 PM, Robert Muir wrote: right, the only way you could really contain it would be to do something like that. I'm looking forward to your ICU analyzer! IMHO, it be great to have it be a pluggable replacement for it's counterparts in core. That is, using reflection, if

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
On Mon, Nov 16, 2009 at 8:17 PM, DM Smith dmsmith...@gmail.com wrote: thanks DM, I hope to work on it more soon... I've been reading the thread and at first my response was. No big deal, it won't affect me (i.e. awareness of the problem). And now my thought is I'm hosed (i.e.

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
well, in all honesty there is a bit of complexity. i leave the StandardTokenizer out of this, it gives the same results regardless of JVM version. it may not be correct, but its consistent, we could wait till 5.0 or 10.0 to make it correct :) Also, because it gives the same results regardless of

Re: Why release 3.0?

2009-11-16 Thread Mark Miller
So whats your best recommendation? Ignoring the difficulty and just considering whats best for users? Robert Muir wrote: well, in all honesty there is a bit of complexity. i leave the StandardTokenizer out of this, it gives the same results regardless of JVM version. it may not be correct,

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
completely ignoring the difficulty, I would propose to fix everything to correspond with the java 1.5 unicode version, for consistency. I would exempt StandardTokenizer, because its completely inside our control. we can fix it at our leisure. for the rest of this stuff, its already a 'change in

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
actually i thought about this. i change my story. deprecating anything is stupid, because its still not back compatible, i.e. Character.isLetter(char) even returns different results now, even if we invoke it. hard break is the only solution. we should have done this deprecation in 2.9, but its

Re: Why release 3.0?

2009-11-16 Thread Robert Muir
That's an amazing number of changes, even when you ignore name changes. DM, for your reference, I created another diff from 4.0-5.1, showing what will happen with JDK7 here: http://people.apache.org/~rmuir/unicodeDiff2.txt the problem is that as a search engine library, lucene cares about