Re: Lucene's default settings back compatibility

2009-06-10 Thread Mark Miller
No one really responded to this Shai? And I take it that the user list never saw it? Perhaps we should just ask for opinion from the user list based on what you already have - just to gauge the reaction on different points. Unless someone responds shortly, we could take a year waiting to

Re: Lucene's default settings back compatibility

2009-06-10 Thread Shai Erera
Well .. to be honest I haven't monitored java-user for quite some time, so I don't know if it hasn't been raised there. But now there's the other thread that Yonik started, so I'm not really sure where to answer. I think that if we look back at 2.0 and compare to 2.9, anyone upgrading from that

Re: Lucene's default settings back compatibility

2009-06-10 Thread Mark Miller
Right - I'd actually hold off now. I figured the threat of sending might prompt some action ;) It still wouldn't hurt to know what the users think, perhaps at more digestible, overview level though. I do think Yonik torpedoed something this liberal :) Thats not a bad thing though. We will

Re: Lucene's default settings back compatibility

2009-05-30 Thread Earwin Burrfoot
As far as I understand the policy-making process, someone from PMC has to start the vote, and then PMC members should, well, vote. Without them taking action we can beep to our hearts' content without any consequences. On Sat, May 30, 2009 at 07:22, Shai Erera ser...@gmail.com wrote: So ... I've

Re: Lucene's default settings back compatibility

2009-05-30 Thread Grant Ingersoll
I think the last piece that is needed is to ask on java-user what others think. In order to do that, I think it needs to be boiled down to a couple paragraphs. -Grant On May 29, 2009, at 11:22 PM, Shai Erera wrote: So ... I've this happen a lot of times (especially in my thesis work) -

Re: Lucene's default settings back compatibility

2009-05-30 Thread Michael McCandless
Actually, I think this is a common, and in fact natural/expected occurrence in open-source. When a tricky topic is discussed, and the opinions are often divergent, frequently the conversation never converges to a consensus and the discussion dies. Only if discussion reaches a semblance of

Re: Lucene's default settings back compatibility

2009-05-30 Thread DM Smith
I think one conclusion that did come of this discussion was that bugs should be fixed even if it breaks backward compatibility. -- DM On May 30, 2009, at 7:21 AM, Michael McCandless wrote: Actually, I think this is a common, and in fact natural/expected occurrence in open-source. When a

Re: Lucene's default settings back compatibility

2009-05-30 Thread Shai Erera
Ok, so digging back in this thread, I think the following proposals were made (if I missed some, please add them): 1. API deprecation last *at least* one full minor release. Example: if we deprecate an API in 2.4, we can remove it in 2.5. BUT, we are also free to keep it there and remove it in

Re: Lucene's default settings back compatibility

2009-05-29 Thread Shai Erera
So ... I've this happen a lot of times (especially in my thesis work) - someone raises a controversial topic, or one that touches the nervous of the system, there's a flurry of activity and then it dies unexpectedly, even though it feels to everyone that there's an extra mile that should be taken

Re: Lucene's default settings back compatibility

2009-05-27 Thread Grant Ingersoll
So, here's a real, concrete example of the need for case by case back compat. See https://issues.apache.org/jira/browse/LUCENE-1662 It's completely stupid that ExtendedFieldCache even exists. It is a dumb workaround for a made up problem that has nothing to do with real coders living in

Re: Lucene's default settings back compatibility

2009-05-25 Thread Michael McCandless
OK thanks Shai! Mike On Mon, May 25, 2009 at 12:18 AM, Shai Erera ser...@gmail.com wrote: Yes - 1630. I'll check 1601 and if nothing's left to do I'll cancel/close it On Sun, May 24, 2009 at 11:25 PM, Michael McCandless luc...@mikemccandless.com wrote: Actually, under LUCENE-1601, what

Re: Lucene's default settings back compatibility

2009-05-24 Thread Shai Erera
One thing I don't fully understand about actsAsVersion (and I know it was said that we may want to drop that approach) - for how long does it stay? I mean, let's take the invalidAcronym. It is a change in back-compat, yes. But for how long are we expected to support it? And if we decide to support

Re: Lucene's default settings back compatibility

2009-05-24 Thread Michael McCandless
On Sun, May 24, 2009 at 2:20 AM, Shai Erera ser...@gmail.com wrote: One thing I don't fully understand about actsAsVersion (and I know it was said that we may want to drop that approach) - for how long does it stay? I mean, let's take the invalidAcronym. It is a change in back-compat, yes. But

Re: Lucene's default settings back compatibility

2009-05-24 Thread Shai Erera
I'm tempted to simply make that change by default for 2.9, now. Agree ! Shai On Sun, May 24, 2009 at 1:28 PM, Michael McCandless luc...@mikemccandless.com wrote: On Sun, May 24, 2009 at 2:20 AM, Shai Erera ser...@gmail.com wrote: One thing I don't fully understand about actsAsVersion

Re: Lucene's default settings back compatibility

2009-05-24 Thread Michael McCandless
I'll open an issue for this and we can discuss under there. And I still need to open issues for the other change defaluts in my original email. Mike On Sun, May 24, 2009 at 8:11 AM, Shai Erera ser...@gmail.com wrote: I'm tempted to simply make that change by default for 2.9, now. Agree !

Re: Lucene's default settings back compatibility

2009-05-24 Thread Shai Erera
I created LUCENE-1601 for that purpose with a fix-version 3.0. I noticed you already opened another issue for the scoring only. So we should remove it from there (note that there's a TODO in the code, if you plan to change it in the new issue you opened). 1601 will still handle the new method

Re: Lucene's default settings back compatibility

2009-05-24 Thread Michael McCandless
Actually, under LUCENE-1601, what more was there to do besides turning off scoring when sorting by field, by default? Is there an issue for adding mating Scorer.scoresDocsInOrder Collector.acceptsDocsOutOfOrder? Mike On Sun, May 24, 2009 at 3:22 PM, Shai Erera ser...@gmail.com wrote: I

Re: Lucene's default settings back compatibility

2009-05-24 Thread Shai Erera
Yes - 1630. I'll check 1601 and if nothing's left to do I'll cancel/close it On Sun, May 24, 2009 at 11:25 PM, Michael McCandless luc...@mikemccandless.com wrote: Actually, under LUCENE-1601, what more was there to do besides turning off scoring when sorting by field, by default? Is there

Re: Lucene's default settings back compatibility

2009-05-22 Thread Earwin Burrfoot
A funny thought: we can give those methods/classes really stupid/nasty names, to emphasize the beauty of the existing API, to encourage people to stick with the better API :) I believe I've seen google using internally names like thisisbadbadbadInstanceMap. :) One thing we didn't address

Re: Lucene's default settings back compatibility

2009-05-22 Thread Matthew Hall
Earwin Burrfoot wrote: As I said, my app uses around ten indexes, which one should I use? :) Even more here, this would be a reasonably painful solution for us. Matt - To unsubscribe, e-mail:

Re: Lucene's default settings back compatibility

2009-05-22 Thread Grant Ingersoll
Perhaps it is wise to take a step back before we play all of these what if games... I think the best way forward is to simply ask ourselves, when confronted with an actual issue, is what is the cost of back compat. for this issue and then address it on a case by case basis, with a bias

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
OK it sounds like a single global actsAsVersion is too problematic. So how about, for cases where back compat default settings are important (analyzers, query scoring changes, etc.) we add actsAsVersion as a mandatory ctor argument to those classes (deprecating the other ctors)? We would do this

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
On Thu, May 21, 2009 at 6:53 PM, Marvin Humphrey mar...@rectangular.com wrote: Lastly, I think a major java Lucene release is justified already. Won't this discussion die down somewhat if you can get 3.0 out? Somewhat, yes, but then when working on 3.1 if we make some great improvement, I'd

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
So, iterating on the proposed changes to back-compat policy: 1. If we deprecate an API in the 2.1 release, we can remove it in the next minor release (2.2). 2. JAR drop-in-ability is only guaranteed on point releases (2.4.1 is a drop-in replacement to 2.4.0). When switching to a

Re: Lucene's default settings back compatibility

2009-05-22 Thread Earwin Burrfoot
 1. If we deprecate an API in the 2.1 release, we can remove it in     the next minor release (2.2). Agree. Maybe also this? 1a. If deprecated functionality is trivially implemented with new one, we reserve the right to delete deprecated things right away with appropriate CHANGES note. Sample I:

Re: Lucene's default settings back compatibility

2009-05-22 Thread Marvin Humphrey
On Fri, May 22, 2009 at 11:53:02AM -0400, Michael McCandless wrote: 1. If we deprecate an API in the 2.1 release, we can remove it in the next minor release (2.2). 2. JAR drop-in-ability is only guaranteed on point releases (2.4.1 is a drop-in replacement to 2.4.0). When

Re: Lucene's default settings back compatibility

2009-05-22 Thread Yonik Seeley
I'm not a lawyer, so I dislike trying to nail down every detail in writing and try to solve future problems in the abstract. Lucene has never really been 100% back compatible... we've just tried to keep it that way... it's more of a mindset than a reality, and I'm wary of changing that mindset

Re: Lucene's default settings back compatibility

2009-05-22 Thread Marvin Humphrey
On Fri, May 22, 2009 at 11:33:33AM -0400, Michael McCandless wrote: when working on 3.1 if we make some great improvement, I'd like new users in 3.1 to see the improvement by default. Sounds like an argument for more frequent major releases. But I'm not exactly one to talk. ;) On

Re: Lucene's default settings back compatibility

2009-05-22 Thread Earwin Burrfoot
In KinoSearch SVN trunk, satellite classes like QueryParser and Highlighter have to be passed a Schema, which contains all the Analyzers.  Analyzers aren't satellite classes under this model -- they are a fixed property of a FullTextType field spec.  Think of them as baked into an SQL field

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
On Fri, May 22, 2009 at 12:52 PM, Marvin Humphrey mar...@rectangular.com wrote: when working on 3.1 if we make some great improvement, I'd like new users in 3.1 to see the improvement by default. Sounds like an argument for more frequent major releases. Yeah. Or rebranding what we now call

Re: Lucene's default settings back compatibility

2009-05-22 Thread Yonik Seeley
On Fri, May 22, 2009 at 1:22 PM, Michael McCandless luc...@mikemccandless.com wrote: (That said, unrelated to this discussion, I would actually like to record per-segment which version of Lucene wrote the segment; this would be very helpful when debugging issues like LUCENE-1474 where I need

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
On Fri, May 22, 2009 at 12:37 PM, Marvin Humphrey mar...@rectangular.com wrote: I still like per-class settings classes. For instance, an IndexWriterSettings class which allows you to hide away all the tweaky stuff that's cluttering up the IndexWriter API. IndexWriterSettings settings =

Re: Lucene's default settings back compatibility

2009-05-22 Thread DM Smith
Yonik Seeley wrote: On Fri, May 22, 2009 at 1:22 PM, Michael McCandless luc...@mikemccandless.com wrote: (That said, unrelated to this discussion, I would actually like to record per-segment which version of Lucene wrote the segment; this would be very helpful when debugging issues like

Re: Lucene's default settings back compatibility

2009-05-22 Thread Marvin Humphrey
I feel the opposite: I'd like new users to see improvements by default, and users that require strict back-compate to ask for that. By strict back-compat, do you mean people who would like their search app to not fail silently? ;) A new user who follows your advice... // haha stupid noob

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
On Fri, May 22, 2009 at 12:44 PM, Yonik Seeley yo...@lucidimagination.com wrote: I'm not a lawyer, so I dislike trying to nail down every detail in writing and try to solve future problems in the abstract. Agreed, and there's always leeway in what we work out here (LUCENE-1436 is a good recent

Re: Lucene's default settings back compatibility

2009-05-22 Thread Marvin Humphrey
On Fri, May 22, 2009 at 01:22:24PM -0400, Michael McCandless wrote: Sounds like an argument for more frequent major releases. Yeah. Or rebranding what we now call minor as major releases, by changing our policy ;) Not sure how much of that is a jest, bug I don't think that's a good idea.

Re: Lucene's default settings back compatibility

2009-05-22 Thread DM Smith
Michael McCandless wrote: On Fri, May 22, 2009 at 12:52 PM, Marvin Humphrey mar...@rectangular.com wrote: when working on 3.1 if we make some great improvement, I'd like new users in 3.1 to see the improvement by default. Sounds like an argument for more frequent major releases.

Re: Lucene's default settings back compatibility

2009-05-22 Thread Marvin Humphrey
On Fri, May 22, 2009 at 09:06:32PM +0400, Earwin Burrfoot wrote: In KinoSearch SVN trunk, satellite classes like QueryParser and Highlighter have to be passed a Schema, which contains all the Analyzers.  Analyzers aren't satellite classes under this model -- they are a fixed property of a

Re: Lucene's default settings back compatibility

2009-05-22 Thread DM Smith
Marvin Humphrey wrote: I feel the opposite: I'd like new users to see improvements by default, and users that require strict back-compate to ask for that. By strict back-compat, do you mean people who would like their search app to not fail silently? ;) A new user who follows your

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
On Fri, May 22, 2009 at 2:27 PM, DM Smith dmsmith...@gmail.com wrote: Marvin Humphrey wrote: I feel the opposite: I'd like new users to see improvements by default, and users that require strict back-compate to ask for that. By strict back-compat, do you mean people who would like their

Re: Lucene's default settings back compatibility

2009-05-22 Thread Earwin Burrfoot
Custom analyzers. No problem. How are they recorded in the index? Several indexes using the same analyzer. No problem.  Only necessary if the analyzer is costly or has some esoteric need for shared state.  And possible via subclassing Schema or Analyzer. It is. Intentionally different

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
OK, net/net it doesn't look like we're going reach agreement on some general approach for having users of Lucene always get the best default settings. We started with the *Settings classes, but that's really a very large project (goes far beyond managing defaults for new users). Then we went to

Re: Lucene's default settings back compatibility

2009-05-22 Thread DM Smith
Michael McCandless wrote: On Fri, May 22, 2009 at 2:27 PM, DM Smith dmsmith...@gmail.com wrote: Marvin Humphrey wrote: I feel the opposite: I'd like new users to see improvements by default, and users that require strict back-compate to ask for that. By strict back-compat,

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
I'd like to do this for 2.9 :) I'll open an issue... (Yes this would just be for diagnostics). Mike On Fri, May 22, 2009 at 1:48 PM, DM Smith dmsmith...@gmail.com wrote: Yonik Seeley wrote: On Fri, May 22, 2009 at 1:22 PM, Michael McCandless luc...@mikemccandless.com wrote: (That said,

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
Well... I would expect hope Lucene's adoption is growing with time, so the number of new users should increase on each release. For a healthy project that's relatively young compared to its potential user base, that growth should be exponential. And, I'd expect the vast majority of old users

Re: Lucene's default settings back compatibility

2009-05-22 Thread DM Smith
Michael McCandless wrote: Well... I would expect hope Lucene's adoption is growing with time, so the number of new users should increase on each release. For a healthy project that's relatively young compared to its potential user base, that growth should be exponential. And, I'd expect the

Re: Lucene's default settings back compatibility

2009-05-22 Thread Marvin Humphrey
On Fri, May 22, 2009 at 10:40:03PM +0400, Earwin Burrfoot wrote: Custom analyzers. No problem. How are they recorded in the index? Analyzers must implement dump() and load(), which convert the Analyzer to/from a JSON-izable data structure. They end up as JSON in index_dir/schema_NNN.json.

Re: Lucene's default settings back compatibility

2009-05-22 Thread Michael McCandless
On Fri, May 22, 2009 at 3:37 PM, DM Smith dmsmith...@gmail.com wrote: So, what is it that they use that leads to such unfavorable results? I think it's simply that they take each search engine, get it to index their collection in the most obvious way, perhaps having read a tutorial somewhere,

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
OK so it sounds like we've boiled the proposal down to two concrete changes to the back-compat policy: 1) Default settings can change; we will always choose defaults based on latest greatest for new users. This only affects runtime behavior. EG in 2.9, when sorting by field you

Re: Lucene's default settings back compatibility

2009-05-21 Thread Shai Erera
I thought that the index file format is supposed to be supported until the 2nd major release. I.e. 3.0 will still read 2.0 indexes, but 4.0 won't. Is that what you meant, or am I wrong? Shai On Thu, May 21, 2009 at 2:17 PM, Michael McCandless luc...@mikemccandless.com wrote: OK so it sounds

Re: Lucene's default settings back compatibility

2009-05-21 Thread DM Smith
On May 21, 2009, at 7:17 AM, Michael McCandless wrote: 1) Default settings can change; we will always choose defaults based on latest greatest for new users. This only affects runtime behavior. EG in 2.9, when sorting by field you won't get scores by default. When we do this

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
On Thu, May 21, 2009 at 8:24 AM, DM Smith dmsmith...@gmail.com wrote: On May 21, 2009, at 7:17 AM, Michael McCandless wrote:  1) Default settings can change; we will always choose defaults based    on latest greatest for new users.  This only affects runtime    behavior.  EG in 2.9, when

Re: Lucene's default settings back compatibility

2009-05-21 Thread Robert Muir
even as simple as changing default stopword list for some analyzer could be an issue, if the user doesn't re-index in response to that change. Can you give an example of such changes? EG if we fix a bug in StandardAnalyzer, we will default it to fixed for new users and expect you on

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
On Thu, May 21, 2009 at 12:19 PM, Robert Muir rcm...@gmail.com wrote: even as simple as changing default stopword list for some analyzer could be an issue, if the user doesn't re-index in response to that change. OK, right. So say we forgot to include the in the default English stopwords list

Re: Lucene's default settings back compatibility

2009-05-21 Thread Matthew Hall
For extreme examples like this, couldn't the stopword list be encapsulated into a single class that's used by the lucene defaults class. That way if you folks released updates to mostly static content like a stopword list, new or old users could get it easily with a simple drop in fix? Just

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
What is the lucene defaults class? Mike On Thu, May 21, 2009 at 12:37 PM, Matthew Hall mh...@informatics.jax.org wrote: For extreme examples like this, couldn't the stopword list be encapsulated into a single class that's used by the lucene defaults class. That way if you folks released

Re: Lucene's default settings back compatibility

2009-05-21 Thread Mark Miller
Michael McCandless wrote: On Thu, May 21, 2009 at 12:19 PM, Robert Muir rcm...@gmail.com wrote: even as simple as changing default stopword list for some analyzer could be an issue, if the user doesn't re-index in response to that change. OK, right. So say we forgot to include the in

Re: Lucene's default settings back compatibility

2009-05-21 Thread DM Smith
Michael McCandless wrote: On Thu, May 21, 2009 at 8:24 AM, DM Smith dmsmith...@gmail.com wrote: On May 21, 2009, at 7:17 AM, Michael McCandless wrote: 1) Default settings can change; we will always choose defaults based on latest greatest for new users. This only affects runtime

Re: Lucene's default settings back compatibility

2009-05-21 Thread Matthew Hall
Sorry, I wasn't quite sure what to call this new class you guys have been talking about. I was referring to the class that's being discussed to encapsulate all of the defaults for a given lucene release. (Its caching strategies etc etc) I'm just not certain that something like a static

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
On Thu, May 21, 2009 at 12:43 PM, Mark Miller markrmil...@gmail.com wrote: Hmmm - thats starting to sound nastier. Its another barrier to upgrading to a new jar. I have to monitor/hunt down and not miss all these little flags so that docs/terms don't disappear from my index? There is already

Re: Lucene's default settings back compatibility

2009-05-21 Thread Robert Muir
yeah, i was thinking the more likely case of where something like teh is in the list... On Thu, May 21, 2009 at 12:25 PM, Michael McCandless luc...@mikemccandless.com wrote: On Thu, May 21, 2009 at 12:19 PM, Robert Muir rcm...@gmail.com wrote: even as simple as changing default stopword list

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
Actually, we started with the *Settings classes (to hold defaults), but then realized a simple actsAsVersion (single static method) would suffice for just the back-compat settings and then pushed further and thought perhaps we should relax our back-compat policy entirely so emulating older

Re: Lucene's default settings back compatibility

2009-05-21 Thread Marvin Humphrey
Mike McCandless: Well this is what I love about the actsAsVersion solution. There's no pain for our back-compat users (besides the one-time effort to set actsAsVersion), and new users always get the best settings. When some mad-as-hell user complains to this list after spending an inordinate

Re: Lucene's default settings back compatibility

2009-05-21 Thread Earwin Burrfoot
That bug has led to 'base' having a compromised reputation among elite users because of intermittent, inexplicable flakiness.  Is that what you want for Lucene? While I agree with that point, Lucene already has lots and lots of static configuration. Having actsAsVersion won't add any new woes.

Re: Lucene's default settings back compatibility

2009-05-21 Thread Jason Rutherglen
I'm having trouble visualizing the various methods people are talking about. It seems like we could open an issue and post patches with code illustrating what each person is talking about? On Thu, May 21, 2009 at 10:02 AM, Michael McCandless luc...@mikemccandless.com wrote: Actually, we

Re: Lucene's default settings back compatibility

2009-05-21 Thread Shai Erera
I thought we were actually on the track towards not introducing any Settings and/or actAs, but instead just change the policy? Can we agree on the following: * Changes to the index file formats need to be supported for 2 major releases. I.e. 2.X indexes need to be read by 3.Y code, but not by

Re: Lucene's default settings back compatibility

2009-05-21 Thread Earwin Burrfoot
Sounds like a good proposition. There's one problem I'd like to address. Good names for classes/members matter, and matter much. They directly affect how fast a newcomer is able to understand that particular API, it also affects how comfortable you work with it once you did understand. When we're

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
On Thu, May 21, 2009 at 4:34 PM, Shai Erera ser...@gmail.com wrote: Changes to the index file formats need to be supported for 2 major releases. I.e. 2.X indexes need to be read by 3.Y code, but not by 4.0. Agreed. Method deprecations last for one full minor release. Your example confused

Re: Lucene's default settings back compatibility

2009-05-21 Thread Earwin Burrfoot
Why not store an actsAs in the index, just for the changes that affect what's in the index?  Ie the index records the version that created it, and by default TokenStreams emulate their behavior as of that version? Because you don't always have access to index at the time you create your

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
On Thu, May 21, 2009 at 1:59 PM, Marvin Humphrey mar...@rectangular.com wrote: That bug has led to 'base' having a compromised reputation among elite users because of intermittent, inexplicable flakiness. Is that what you want for Lucene? While I agree a single global default is not great, I

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
On Thu, May 21, 2009 at 5:19 PM, Earwin Burrfoot ear...@gmail.com wrote: Why not store an actsAs in the index, just for the changes that affect what's in the index?  Ie the index records the version that created it, and by default TokenStreams emulate their behavior as of that version?

Re: Lucene's default settings back compatibility

2009-05-21 Thread Robert Muir
and what if your analyzer needs a third-party library (or two)? i mean this isn't unique to analyzers, if something changes/bug is fixed in the guts of some query/scorer that affects scoring in the slightest then thats a potential issue too, right? for a big index burying a result deep is

Re: Lucene's default settings back compatibility

2009-05-21 Thread Michael McCandless
On Thu, May 21, 2009 at 5:44 PM, Robert Muir rcm...@gmail.com wrote: and what if your analyzer needs a third-party library (or two)? In such cases the back-compat of your analyzer is your responsibility, right? i mean this isn't unique to analyzers, if something changes/bug is fixed in the

Re: Lucene's default settings back compatibility

2009-05-21 Thread Robert Muir
On Thu, May 21, 2009 at 5:55 PM, Michael McCandless luc...@mikemccandless.com wrote: On Thu, May 21, 2009 at 5:44 PM, Robert Muir rcm...@gmail.com wrote: and what if your analyzer needs a third-party library (or two)? In such cases the back-compat of your analyzer is your responsibility,

RE: Lucene's default settings back compatibility

2009-05-21 Thread Steven A Rowe
On 5/21/2009 at 7:17 AM, Michael McCandless wrote: OK so it sounds like we've boiled the proposal down to two concrete changes to the back-compat policy: 1) Default settings can change; we will always choose defaults based on latest greatest for new users. This only affects

Re: Lucene's default settings back compatibility

2009-05-21 Thread Marvin Humphrey
On Thu, May 21, 2009 at 05:19:43PM -0400, Michael McCandless wrote: Marvin, which solution would you prefer? Between the two, I'd prefer settings constructor arguments, though I would be inclined to have settings classes that are specific to individual classes rather than Lucene-wide. At

Re: Lucene's default settings back compatibility

2009-05-21 Thread Shai Erera
Your example confused me. You're right. I Wrote it with one eye closed already. I meant to say that if I'm a 2.4 user and something gets deprecated in trunk (afterwards), it is carried through 2.4.X and 2.5 and then removed in 2.6. So only 1 full minor release. It's somewhat crazy, but what

Re: Lucene's default settings back compatibility

2009-05-20 Thread Grant Ingersoll
On May 19, 2009, at 1:51 PM, Michael McCandless wrote: I think you've moved onto discussing something different: should we relax our back compat policy. I'm all for that discussion, but it's different from given our back compat policy, how can we implement it w/o harming new users of Lucene.

Re: Lucene's default settings back compatibility

2009-05-20 Thread Michael McCandless
On Tue, May 19, 2009 at 4:50 PM, Yonik Seeley yo...@lucidimagination.com wrote: Right, that's exactly why I want to fix it (only one behavior allowed and so for all of 2.* we must match the 2.0 behavior). I meant one jar per per-jvm gives you one behavior (as is the case now). But by setting

Re: Lucene's default settings back compatibility

2009-05-20 Thread Mark Miller
Michael McCandless wrote: On Tue, May 19, 2009 at 4:50 PM, Yonik Seeley yo...@lucidimagination.com wrote: Right, that's exactly why I want to fix it (only one behavior allowed and so for all of 2.* we must match the 2.0 behavior). I meant one jar per per-jvm gives you one behavior

Re: Lucene's default settings back compatibility

2009-05-20 Thread Yonik Seeley
On Wed, May 20, 2009 at 7:22 AM, Michael McCandless luc...@mikemccandless.com wrote: So I think you're suggesting something like this: when you use Lucene, if you want latest and greatest defaults, do nothing. If instead you want defaults to match a particular past minor release, you must

Re: Lucene's default settings back compatibility

2009-05-20 Thread DM Smith
I like the idea of settings however it is implemented. With the blurring of core and contrib in the repackaging of Lucene, the issue of backward compatibility becomes more difficult. (maybe, I'm imagining problems where they don't exist.) My concern with any of these mechanisms: codifying

Re: Lucene's default settings back compatibility

2009-05-20 Thread Michael McCandless
On Wed, May 20, 2009 at 8:28 AM, Yonik Seeley yo...@lucidimagination.com wrote: So I think you're suggesting something like this: when you use Lucene, if you want latest and greatest defaults, do nothing. If instead you want defaults to match a particular past minor release, you must call

Re: Lucene's default settings back compatibility

2009-05-20 Thread Marvin Humphrey
But since 3.0 is a major release anyway, we could change the default of actsAsVersion with each 3.x release (or just set it to 3) and require that a users set actsAsVersion=3 (or whatever version they are on) in order to get maximum back compatibility. What happens when two libraries

Re: Lucene's default settings back compatibility

2009-05-20 Thread Earwin Burrfoot
Exactly what happens when you call BooleanQuery.setMaxClauseCount(n) from two libraries. Last one wins. On Wed, May 20, 2009 at 17:50, Marvin Humphrey mar...@rectangular.com wrote: But since 3.0 is a major release anyway, we could change the default of actsAsVersion with each 3.x release (or

Re: Lucene's default settings back compatibility

2009-05-20 Thread Marvin Humphrey
On Wed, May 20, 2009 at 05:57:49PM +0400, Earwin Burrfoot wrote: What happens when two libraries loaded in the same VM have Lucene as a dependency and set actsAsVersion to conflicting numbers? Exactly what happens when you call BooleanQuery.setMaxClauseCount(n) from two libraries. Last

Re: Lucene's default settings back compatibility

2009-05-20 Thread Mark Miller
Marvin Humphrey wrote: Yeesh, that's evil. :( It will be sweet, sweet justice if one of your own projects gets infected by the kind of action-at-a-distance bug you're so blithely unconcerned about Heh. Thats a bit over the top. It is evil stuff, but its much less evil in this very contained

Re: Lucene's default settings back compatibility

2009-05-20 Thread Andi Vajda
On Wed, 20 May 2009, Michael McCandless wrote: On Wed, May 20, 2009 at 11:57 AM, Yonik Seeley yo...@lucidimagination.com wrote: On Wed, May 20, 2009 at 11:46 AM, Mark Miller markrmil...@gmail.com wrote: Marvin Humphrey wrote: Yeesh, that's evil.  :( It will be sweet, sweet justice if one

Re: Lucene's default settings back compatibility

2009-05-20 Thread Michael McCandless
On Wed, May 20, 2009 at 11:57 AM, Yonik Seeley yo...@lucidimagination.com wrote: On Wed, May 20, 2009 at 11:46 AM, Mark Miller markrmil...@gmail.com wrote: Marvin Humphrey wrote: Yeesh, that's evil.  :( It will be sweet, sweet justice if one of your own projects gets infected by the kind

Re: Lucene's default settings back compatibility

2009-05-20 Thread Michael McCandless
On Wed, May 20, 2009 at 12:55 PM, Andi Vajda va...@osafoundation.org wrote: I've been watching this thread with interest with my opinion swaying back and forth. So have I! This last comment, though, pushes me to favor the settings class idea because that idea came with the promise of

Re: Lucene's default settings back compatibility

2009-05-20 Thread Yonik Seeley
On Wed, May 20, 2009 at 11:46 AM, Mark Miller markrmil...@gmail.com wrote: Marvin Humphrey wrote: Yeesh, that's evil.  :( It will be sweet, sweet justice if one of your own projects gets infected by the kind of action-at-a-distance bug you're so blithely unconcerned about Heh. Thats a bit

Re: Lucene's default settings back compatibility

2009-05-20 Thread Shai Erera
Then why go through all this trouble and not simply change the back-compat policy? Really, I read some of Grant's responses and I realize that I've upgraded to 2.4 way too long ago. 2.9 is nowhere in sight. It takes a lot of time to release and during that time there's lots of discussions on the

Re: Lucene's default settings back compatibility

2009-05-20 Thread Earwin Burrfoot
In fact, there's no reason to upgrade Lucene (save for bigfixes), if you absolutely require a drop-in jar, and don't want to touch any of your code. See, you upgrade either for new features, or for performance improvements. You have to write code for former, and you have to write code for the

Re: Lucene's default settings back compatibility

2009-05-20 Thread Mark Miller
Earwin Burrfoot wrote: See, you upgrade either for new features, or for performance improvements. You have to write code for former, and you have to write code for the latter (because by default most of them are off). Thats not completely true. If you have upgraded Lucene over the years and

Re: Lucene's default settings back compatibility

2009-05-20 Thread Shai Erera
Exactly ! which is why I think we should relax the back-compat policy a bit. And ... (I realize it's going to complicate things a bit) we could also decide to have dot release for bug fixes, like we had 2.4.1. So let's say when 3.4 comes (3-4 years from now :) ). In 3.6 we don't preserve any

Re: Lucene's default settings back compatibility

2009-05-20 Thread Earwin Burrfoot
Mark Miller: If you have upgraded Lucene over the years and you never touched code to tweak performance, you still got fantastic performance improvements. You just didn't get them all. If you never touched the code over the years, your project is probably already dead. Shai Erera: Exactly !

Re: Lucene's default settings back compatibility

2009-05-20 Thread Michael McCandless
On Wed, May 20, 2009 at 3:24 PM, Shai Erera ser...@gmail.com wrote: Then why go through all this trouble and not simply change the back-compat policy? OK so let's talk policy now ;) We need some serious relaxing of the back-compat policy to make the actsAsVersion proposal pointless. Ie

Re: Lucene's default settings back compatibility

2009-05-20 Thread Shai Erera
Earwin - I wrote it before - index structure is the only back-compat policy I propose to keep, and not for just one major release, but for 2 (which I believe is the current behavior already). I also absolutely don't want to give that up. I think it's not unreasonable to say if you want to take

Re: Lucene's default settings back compatibility

2009-05-20 Thread Mark Miller
Earwin Burrfoot wrote: See, you upgrade either for new features, or for performance improvements. You have to write code for former, and you have to write code for the latter (because by default most of them are off). Mark Miller: If you have upgraded Lucene over the years and you never

Re: Lucene's default settings back compatibility

2009-05-20 Thread Mark Miller
Shai Erera wrote: +1 (am I allowed to cast +1s not being a committer?) :) Absolutely :) When push comes to shove, you don't even have a valid vote as a Committer. Only members of the PMC have binding votes. You have as much voting power as a committer as long as you have as much an ability

  1   2   >