There are already some proposals of reforming the whole Document/Field API because it does not match a Full Text Search engine using an Inverted Index. Stored fields and indexed fields should not be mixed together. The problem then disappears, because you are forced to split between indexing and storing so you have to take care in separate about storing and indexing.
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Glen Newton [mailto:glen.new...@gmail.com] > Sent: Wednesday, November 18, 2009 3:24 PM > To: java-user@lucene.apache.org > Subject: Re: Lucene Java 3.0.0 RC1 now available for testing > > Yes, I would agree with you on the surprise aspect. :-) > > But you suggest hiding complexity, and being in control and having > transparency are mutually exclusive, which isn't necesarily the case. > > I think I can live with the decisions made. :-) > If I can think of a viable and complete alternative, I'll run it by > the community. > > thanks, > Glen > > 2009/11/18 Otis Gospodnetic <otis_gospodne...@yahoo.com>: > > Well, I think some people will be for hiding complexity, while others > will be for being in control and having transparency. Think how surprised > one would be to find 1 extra field in his index, say when looking at their > index with Luke. :) > > Otis > > -- > > Sematext is hiring -- http://sematext.com/about/jobs.html?mls > > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR > > > > > > > > ----- Original Message ---- > >> From: Glen Newton <glen.new...@gmail.com> > >> To: java-user@lucene.apache.org > >> Sent: Tue, November 17, 2009 10:53:01 PM > >> Subject: Re: Lucene Java 3.0.0 RC1 now available for testing > >> > >> I understand the reasons, but - if I may ask so late in the game - was > >> this the best way to do this? > >> > >> From a user (developer) perspective, this is an implementation issue. > >> Couldn't this have been done behind the scenes, so that when I asked > >> for Field.Index.ANALYZED && Field.Store.COMPRESS, instead of what > >> previously happened (and was variously problematic), two fields were > >> transparently created, one being binary compressed stored and the > >> other being indexed only? The Field API could hide all of this > >> complexity, using one underlying Field when I use Field.getString() > >> (compressed stored one), using the other when I use Field.setBoost() > >> (the indexed one) and both when I call Field.setValue(). This might > >> have less impact on developers and be less disruptive on API changes. > >> Oh, some naming convention could handle the underlying Fields. > >> > >> A little complicated I agree. > >> > >> Again, apologies to those who worked hard on these changes: my fault > >> for not noticing this sooner (I hadn't started moving my code to 2.9 > >> from 2.4 so I hadn't read the deprecation signs). > >> > >> thanks, > >> > >> Glen > >> > >> > >> > >> 2009/11/17 Mark Miller : > >> > Here is some of the history: > >> > > >> > https://issues.apache.org/jira/browse/LUCENE-652 > >> > https://issues.apache.org/jira/browse/LUCENE-1960 > >> > > >> > Glen Newton wrote: > >> >> Could someone send me where the rationale for the removal of > >> >> COMPRESSED fields is? I've looked at > >> >> > >> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0- > rc1/changes/Changes.html#3.0.0.changes_in_runtime_behavior > >> >> but it is a little light on the 'why' of this change. > >> >> > >> >> My fault - of course - for not paying attention. > >> >> > >> >> thanks, > >> >> Glen > >> >> > >> >> 2009/11/17 Uwe Schindler : > >> >> > >> >>> Hello Lucene users, > >> >>> > >> >>> > >> >>> > >> >>> On behalf of the Lucene dev community (a growing community far > larger than > >> >>> just the committers) I would like to announce the first release > candidate > >> >>> for Lucene Java 3.0. > >> >>> > >> >>> > >> >>> > >> >>> Please download and check it out - take it for a spin and kick the > tires. If > >> >>> all goes well, we hope to release the final version of Lucene 3.0 > in a > >> >>> little over a week. > >> >>> > >> >>> > >> >>> > >> >>> The new version is mostly a cleanup release without any new > features. All > >> >>> deprecations targeted to be removed in version 3.0 were removed. If > you are > >> >>> upgrading from version 2.9.1 of Lucene, you have to fix all > deprecation > >> >>> warnings in your code base to be able to recompile against this > version. > >> >>> > >> >>> > >> >>> > >> >>> This is the first Lucene release with Java 5 as a minimum > requirement. The > >> >>> API was cleaned up to make use of Java 5's generics, varargs, > enums, and > >> >>> autoboxing. New users of Lucene are advised to use this version for > new > >> >>> developments, because it has a clean, type safe new API. Upgrading > users can > >> >>> now remove unnecessary casts and add generics to their code, too. > If you > >> >>> have not upgraded your installation to Java 5, please read the file > >> >>> JRE_VERSION_MIGRATION.txt (please note that this is not related to > Lucene > >> >>> 3.0, it will also happen with any previous release when you upgrade > your > >> >>> Java environment). > >> >>> > >> >>> > >> >>> > >> >>> Lucene 3.0 has some changes regarding compressed fields: 2.9 > already > >> >>> deprecated compressed fields; support for them was removed now. > Lucene 3.0 > >> >>> is still able to read indexes with compressed fields, but as soon > as merges > >> >>> occur or the index is optimized, all compressed fields are > decompressed and > >> >>> converted to Field.Store.YES. Because of this, indexes with > compressed > >> >>> fields can suddenly get larger. > >> >>> > >> >>> > >> >>> > >> >>> While we generally try and maintain full backwards compatibility > between > >> >>> major versions, Lucene 3.0 has some minor breaks, mostly related to > >> >>> deprecation removal, pointed out in the 'Changes in backwards > compatibility > >> >>> policy' section of CHANGES.txt. Notable are: > >> >>> > >> >>> > >> >>> > >> >>> - IndexReader.open(Directory) now opens in read-only mode per > default (this > >> >>> method was deprecated because of that in 2.9). The same occurs to > >> >>> IndexSearcher. > >> >>> > >> >>> - Already started in 2.9, core TokenStreams are now made final to > enforce > >> >>> the decorator pattern. > >> >>> > >> >>> - If you interrupt an IndexWriter merge thread, IndexWriter now > throws an > >> >>> unchecked ThreadInterruptedException that extends RuntimeException > and > >> >>> clears the interrupt status. > >> >>> > >> >>> > >> >>> > >> >>> Also, remember that this is a release candidate, and not the final > Lucene > >> >>> 3.0 release. > >> >>> > >> >>> > >> >>> > >> >>> You can find the full list of changes here: > >> >>> > >> >>> > >> >>> > >> >>> HTML version: > >> >>> > >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0- > rc1/changes/C > >> >>> hanges.html > >> >>> > >> >>> > >> >>> > >> >>> Text version: > >> >>> > >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0- > rc1/changes/C > >> >>> hanges.txt > >> >>> > >> >>> > >> >>> > >> >>> Changes have also occurred in Lucene's contrib area: > >> >>> > >> >>> > >> >>> > >> >>> HTML version: > >> >>> > >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0- > rc1/changes/C > >> >>> ontrib-Changes.html > >> >>> > >> >>> > >> >>> > >> >>> Text version: > >> >>> > >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0- > rc1/changes/C > >> >>> ontrib-Changes.txt > >> >>> > >> >>> > >> >>> > >> >>> Download release candidate 1 here: > >> >>> > >> >>> http://people.apache.org/~uschindler/staging-area/lucene-3.0.0-rc1/ > >> >>> > >> >>> > >> >>> > >> >>> Be sure to report back with any issues you find! Look especially > for faults > >> >>> in generification of public APIs (like missing wildcards,...). > >> >>> > >> >>> > >> >>> > >> >>> Thanks, > >> >>> > >> >>> Uwe Schindler > >> >>> > >> >>> > >> >>> > >> >>> ----- > >> >>> > >> >>> Uwe Schindler > >> >>> > >> >>> H.-H.-Meier-Allee 63, D-28213 Bremen > >> >>> > >> >>> http://www.thetaphi.de > >> >>> > >> >>> eMail: u...@thetaphi.de > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >> > >> >> > >> >> > >> >> > >> > > >> > > >> > -- > >> > - Mark > >> > > >> > http://www.lucidimagination.com > >> > > >> > > >> > > >> > > >> > --------------------------------------------------------------------- > >> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >> > For additional commands, e-mail: java-user-h...@lucene.apache.org > >> > > >> > > >> > >> > >> > >> -- > >> > >> - > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > > > -- > > - > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
