Hello,
I am using ANTLR 3.2.
With a lexer grammar such as this one:
lexer grammar Z;
options {
filter=true;
}
IS_KEYWORD : 'IS' {skip();};
The skip() is ignored and the token gets inserted in the token stream.
If I change to filter=false, skip is honored.
The problem might come from the nextToken() method which seems to be
overriden in the generated lexer code when filter=true.
The nextToken() method in org.antlr.runtime.Lexer has code like this:
try {
mTokens();
if ( state.token==null ) {
emit();
}
else if ( state.token==Token.SKIP_TOKEN ) {
continue;
}
return state.token;
}
While the generated equivalent code (in Z.java) looks like this:
try {
int m = input.mark();
state.backtracking=1;
state.failed=false;
mTokens();
state.backtracking=0;
if ( state.failed ) {
input.rewind(m);
input.consume();
}
else {
emit();
return state.token;
}
}
So whatever the state.token value, the token is emitted.
Or I messed up and there some other explanation?
Thanks for your help.
Fady
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---