[
https://issues.apache.org/jira/browse/LUCENE-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784788#action_12784788
]
Uwe Schindler edited comment on LUCENE-2074 at 12/2/09 12:12 PM:
-----------------------------------------------------------------
I tried with JFlex 1.5 svn trunk version from today, built with maven. The
produced java file is almost identical, the big array is 100% identical. There
are only some new private methods and one switch was reordered. The interface
works for both versions, so nothing to change.
I am not sure, which unicode version this JFlex now uses and if Steven's
changes are in it. I cannot test with Java 1.4, as this version requires 1.5.
I attached a patch with the difference between the JFlex-1.4.1 on Java 5
generated java file and the snapshot version (also Java 5). You can see that
the big DFA array was not changed at all.
was (Author: thetaphi):
I tried with JFlex 1.5 svn trunk version from today, built with maven. The
produced java file is almost identical, the big array is 100% identical. There
are only some new private methods and one switch was reordered. The interface
works for both versions, so nothing to change.
I am not sure, which unicode version this JFlex now uses and if Steven's
changes are in it. I cannot test with Java 1.4, as this version requires 1.5.
I attached a patch with the difference between the 1.4.1 generated java file
and the snapshot version. You can see that the big DFA array was not changed at
all.
> Use a separate JFlex generated Unicode 4 by Java 5 compatible
> StandardTokenizer
> -------------------------------------------------------------------------------
>
> Key: LUCENE-2074
> URL: https://issues.apache.org/jira/browse/LUCENE-2074
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 3.1
>
> Attachments: jflex-1.4.1-vs-1.5-snapshot.diff, jflexwarning.patch,
> LUCENE-2074-lucene30.patch, LUCENE-2074.patch, LUCENE-2074.patch,
> LUCENE-2074.patch, LUCENE-2074.patch
>
>
> The current trunk version of StandardTokenizerImpl was generated by Java 1.4
> (according to the warning). In Java 3.0 we switch to Java 1.5, so we should
> regenerate the file.
> After regeneration the Tokenizer behaves different for some characters.
> Because of that we should only use the new TokenizerImpl when
> Version.LUCENE_30 or LUCENE_31 is used as matchVersion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
