push :) Cheers,
Trond On Fri, Nov 16, 2012 at 5:35 PM, Lubos Kosco <[email protected]> wrote: > > If no one has any objections, I will push this > > - I miss maven build scripts, but I will fix them in separate changeset > (and I miss eclipse project settings, but I will skip this too for now) > > next step will be obviously make the lucene compatibility class work (so > make analyzers 100% compatible, currently they work because of our own > "analyze" method used and cannot be chained - so a redesign will be in > place) > > cheers > L > > > On 14.11.2012 17:17, Lubos Kosco wrote: > >> >> Now with all the changes : >> https://cr.opensolaris.org/**action/browse/opengrok/taz/** >> lucene40/webrev-2012-11-14-**lucene_40/<https://cr.opensolaris.org/action/browse/opengrok/taz/lucene40/webrev-2012-11-14-lucene_40/> >> >> and without nb7.2 upgrade changes >> - depends on JDK1.7 >> - I added a fix for NPE when webapp is not inited (thnx Lukas) >> - source code formatting and cleanups, license, date fixes >> - build fixes >> - package fixes >> - lucene compatibility class will fail however - we need to redesign the >> Analyzers, ev. get rid of cache-ing the whole file into memory to be >> compatible with test class (next step) >> >> thnx >> Lubos >> >> On 13.11.2012 13:14, Lubos Kosco wrote: >> >>> On 13.11.2012 11:18, Knut Anders Hatlen wrote: >>> >>>> Lubos Kosco <[email protected]> writes: >>>> >>>> Hi guys, can you have a look please? >>>>> >>>> Thanks, Lubos. I had a quick look at the patch (skipped quickly past the >>>> >>> >>> Thank you for the review kah ;) >>> >>> big changes to the NetBeans project files...), and it looks mostly fine >>>> to me. I'm not a Lucene expert, though. >>>> >>>> Some questions: >>>> >>>> I saw many changes like this one: >>>> >>>> - } >>>> - doc.add(new Field("date", date, Field.Store.YES, >>>> Field.Index.NOT_ANALYZED)); >>>> + } >>>> + doc.add(new Field("date", date, StringField.TYPE_STORED)); >>>> >>>> According to >>>> http://lucene.apache.org/core/**4_0_0/MIGRATE.html<http://lucene.apache.org/core/4_0_0/MIGRATE.html>, >>>> one >>>> should also call ft.setOmitNorms(false) to preserve the original >>>> semantics for this particular combination of arguments. Is this >>>> something we need to do? >>>> >>> >>> good catch, I had it there before, so it's my mistake when doing >>> merges(I had some fighting with the reusable components in Analyzers :( , >>> ev. ApacheCon visit solved it :-D ), will check all fieldtypes used >>> >>> >>>> There are also many new instance variables in the analyzers (for the >>>> TokenStreamComponents and tokenizers used in the new createComponents() >>>> methods), but I don't see that they are ever read except in the local >>>> scope where they are assigned a value. Could they be changed to local >>>> variables? >>>> >>> >>> I guess yes, they are cached by Analyzer APIs after created once and all >>> subsequent tokenStream calls reuse the cache, or create the object(s) >>> I will change where appropriate and no other reuse occurs >>> >>> I also forgot to do formatting, so I will run autoformat from NB on all >>> changed files >>> >>> >>>> Why? >>>>> Lucene 4.0 is 300% faster when indexing, 100% faster for queries >>>>> >>>> Sounds promising! :) >>>> >>>> We can easily add regexp queries now >>>>> We can easily take latest highlighting >>>>> index statistics (and all what can be done from them, better >>>>> searching, grouping, categorization/classification) >>>>> possible Solr/Tika integrations >>>>> (for more search the web please) >>>>> >>>> >>> this above should be more promising, I actually hope for highlighter and >>> some index statistics to be used soon >>> also I have the tika for pdf and open/libre/ms office integration >>> pending, once we're done with lucene4.0 ;) >>> >>> >>>>> Webrev >>>>> http://stargate.cnl.tuke.sk/~**taz/webrev-2012-11-09-lucene_**40/<http://stargate.cnl.tuke.sk/~taz/webrev-2012-11-09-lucene_40/> >>>>> >>>>> not tested(in progress): >>>>> updating documents (I am not sure on uid document retrieval, wasn't >>>>> obvious to port) >>>>> >>>>> tested: all junits, regression on term count numbers (same numbers of >>>>> tokenized terms as in 3.6.1) >>>>> >>>>> what's missing: build system fixes so l40 will get autodownloaded , >>>>> >>>> Another build problem I had, was that I had to set the >>>> platforms.JDK_1.7.home variable when running Ant. Never seen that >>>> before. It works fine without the patch. >>>> >>>> BUILD FAILED >>>> /code/opengrok/trunk/**nbproject/build-impl.xml:86: The J2SE Platform >>>> is not correctly set up. >>>> Your active platform is: JDK_1.7, but the corresponding property >>>> "platforms.JDK_1.7.home" is not found in the project's properties files. >>>> Either open the project in the IDE and setup the Platform with the >>>> same name or add it manually. >>>> For example like this: >>>> ant -Duser.properties.file=<path_**to_property_file> jar (where >>>> you put the property "platforms.JDK_1.7.home" in a .properties file) >>>> or ant -Dplatforms.JDK_1.7.home=<**path_to_JDK_home> jar (where no >>>> properties file is used) >>>> >>> >>> yes, I haven't played with build system yet and netbeans was upgraded in >>> my env in between too(which seems like the cause) >>> so I won't push anything until I test both ant in cli and from netbeans >>> (if I will have spare time I might do the same for eclipse and idea - >>> saw on bitbucket that J. Ryan Stinnett was developing OpenGrok in Idea, >>> what is a very good idea ;) ) >>> >>> >>>> >>>> package-ing fixes, lucene compatibility test auto run if lucene >>>>> test-framework on classpath >>>>> (will add it once I get some time) >>>>> >>>> >>> I have the lucene compatibility tests running (in a very basic form, but >>> they are) - when the classpath has the lucene test-framework jars >>> >>> will publish new review today/tomorrow >>> >>> cheers >>> Lubos >>> >>> ______________________________**_________________ >>> opengrok-dev mailing list >>> [email protected] >>> http://mail.opensolaris.org/**mailman/listinfo/opengrok-dev<http://mail.opensolaris.org/mailman/listinfo/opengrok-dev> >>> >> >> > -- Trond Norbye
_______________________________________________ opengrok-dev mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/opengrok-dev
