At 09:16 27/01/2009, ian eyberg wrote: >Running this through antlrworks I am hitting my 'chatrule' >every time but it will start chopping up the >player's message into other tokens that I have defined. > >My question: why wouldn't the ALPHA token always be >chosen over the other tokens?
Lexing is done independently of parsing (and before parsing begins). Essentially, when the lexer is "between" tokens it will look at the input and decide which of the lexer rules best fits the input. Then it will consume input using that rule until the rule is no longer satisfied, and then it will exit out and look for another lexer rule to satisfy. When you have overlapping definitions (such as ALPHA and LAMB), this can lead to differences in behaviour depending on the ordering of the input. (eg. "rack" => LAMB, "racket" => LAMB ALPHA, "track" => ALPHA) List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en -~----------~----~----~----~------~----~------~--~---
