[ https://issues.apache.org/jira/browse/LUCENE-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12778598#action_12778598 ]
Uwe Schindler edited comment on LUCENE-2074 at 11/16/09 10:18 PM: ------------------------------------------------------------------ It uses hardcode char ranges, the parser is therefore not JDK-dependent. Let's keep it as it is for now. Mediawiki itsself is not unicode conform, because it's written in PHP and PHP only gets unicode in 6.0 (*lol*) (that says a PHP core committer named U.S. from Bremen in Germany...) was (Author: thetaphi): It uses hardcode char ranges, the parser is therefore not JDK-dependent. Let's keep it as it is for now. Mediawiki itsself is not unicode conform, because it's written in PHP and PHP only gets unicode in 6.0 *lol* (that says a PHP core committer names U.S. from Bremen in Germany...) > Use a separate JFlex generated Unicode 4 by Java 5 compatible > StandardTokenizer > ------------------------------------------------------------------------------- > > Key: LUCENE-2074 > URL: https://issues.apache.org/jira/browse/LUCENE-2074 > Project: Lucene - Java > Issue Type: Bug > Affects Versions: 3.0 > Reporter: Uwe Schindler > Assignee: Uwe Schindler > Fix For: 3.0 > > Attachments: jflexwarning.patch, LUCENE-2074.patch, LUCENE-2074.patch > > > The current trunk version of StandardTokenizerImpl was generated by Java 1.4 > (according to the warning). In Java 3.0 we switch to Java 1.5, so we should > regenerate the file. > After regeneration the Tokenizer behaves different for some characters. > Because of that we should only use the new TokenizerImpl when > Version.LUCENE_30 is used as matchVersion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org