Yes, you can change the rules and recompile. Before you recompile, you have to run 'ant jflex' to generate the java source.
Steve -----Original Message----- From: Dmitry Kan [mailto:dmitry....@gmail.com] Sent: Wednesday, July 06, 2011 9:21 AM To: solr-user@lucene.apache.org Subject: Re: solr.StandardTokenizerFactory: more info needed Hi Steven, This looks very good. Thanks. Do I understand correctly, that I were to change the tokenizer rules, I could go and change e.g. the token class definitions (like <NUM>) in this file and recompile the code? On Wed, Jul 6, 2011 at 3:45 PM, Steven A Rowe <sar...@syr.edu> wrote: > Hi Dmitry, > > The underlying Lucene implementation is here: > http://svn.apache.org/repos/asf/lucene/java/tags/lucene_2_9_1/src/java > /org/apache/lucene/analysis/standard/ > > StandardTokenizerImpl.jflex is probably where you should start. > > Steve > > -----Original Message----- > From: Dmitry Kan [mailto:dmitry....@gmail.com] > Sent: Wednesday, July 06, 2011 3:23 AM > To: solr-user@lucene.apache.org > Subject: solr.StandardTokenizerFactory: more info needed > > Hi all! > > solr.StandardTokenizerFactory -- is it possible to see the full > description of its behaviour for solr.1.4 somewhere? Wiki > http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.Stand > ardTokenizerFactory > is > very short.. > > -- > Regards, > > Dmitry Kan > -- Regards, Dmitry Kan
