Hi Dmitry, The underlying Lucene implementation is here: http://svn.apache.org/repos/asf/lucene/java/tags/lucene_2_9_1/src/java/org/apache/lucene/analysis/standard/
StandardTokenizerImpl.jflex is probably where you should start. Steve -----Original Message----- From: Dmitry Kan [mailto:dmitry....@gmail.com] Sent: Wednesday, July 06, 2011 3:23 AM To: solr-user@lucene.apache.org Subject: solr.StandardTokenizerFactory: more info needed Hi all! solr.StandardTokenizerFactory -- is it possible to see the full description of its behaviour for solr.1.4 somewhere? Wiki http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.StandardTokenizerFactory is very short.. -- Regards, Dmitry Kan
