Hi,
In message "Re: [kaffe] Bug report (java.io.StreamTokenizer)"
on 03/06/30, Hermanni Hyytiälä <[EMAIL PROTECTED]> writes:
> According to the JLS (first edition), the nextToken-method of
> java.io.StreamTokenizer class has the following lexical order:
>
> whitespace
> numeric character
> alphabetic character
> comment character
> string quote character
> comment //
> comment /*
I see. But this rule seems to have been ignored even by
Sun's implementation.
(1) Kaffe's java.io.StreamTokenizer.java has this comment:
/* Contrary to the description in JLS 1.ed,
C & C++ comments seem to be checked
before other comments. That actually
make sense, since the default comment
character is '/'.
*/
(2) Comment characters must be checked before alphabetic characters
and Sun's java.io.StreamTokenizer seems to do so. Otherwise,
NBIO you mentioned cannot run properly.
> Yet, however, I haven't tested if this parsing order changes the
> behaviour of Kaffe's java.io.StringTokenizer's behaviour (could someone
> test it ?-)).
I wrote a simple program to test with Sun's JDK and kaffe with and
without my patch.
import java.io.*;
public class StreamTokenizerTest {
public static void main(String[] args) throws Exception {
StreamTokenizer tok = new StreamTokenizer(System.in);
if (args[0].equals("NBIO")) {
tok = new StreamTokenizer(System.in);
tok.resetSyntax();
tok.wordChars((char)0, (char)255);
tok.whitespaceChars('\u0000', '\u0020');
tok.commentChar('#');
tok.eolIsSignificant(true);
}
System.out.println("TT_WORD = " + StreamTokenizer.TT_WORD);
System.out.println("TT_NUMBER = " + StreamTokenizer.TT_NUMBER);
System.out.println("TT_EOF = " + StreamTokenizer.TT_EOL);
System.out.println("TT_EOF = " + StreamTokenizer.TT_EOF);
while (true) {
int t = tok.nextToken();
System.out.println(tok.sval + ": " + t);
if (t == StreamTokenizer.TT_EOF) break;
}
}
}
Sun's JDK:
bash$ echo "# comment" |java StreamTokenizerTest NBIO
TT_WORD = -3
TT_NUMBER = -2
TT_EOF = 10
TT_EOF = -1
null: 10
null: -1
Kaffe without my patch:
$ echo "# comment" |java StreamTokenizerTest NBIO
TT_WORD = -3
TT_NUMBER = -2
TT_EOF = 10
TT_EOF = -1
#: -3
comment: -3
comment: 10
null: -1
Kaffe with my patch gives the same result as Sun's.
$ echo "# comment" |java StreamTokenizerTest NBIO
TT_WORD = -3
TT_NUMBER = -2
TT_EOF = 10
TT_EOF = -1
null: 10
null: -1
_______________________________________________
kaffe mailing list
[EMAIL PROTECTED]
http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
