Sylvain, Gregory [USA] wrote:
Hello all,
 
Looking for information about best practice, performance and reasoning with respect to using literals in the parser verses using tokens in the parser.   For example, if you know you will have a phrase of the language you are trying to parse begin with 'CONTENT/', which is better:
 
Variation A:
 
r    : 'CONTENT/' <rest> ;
 
 
Variation B:
 
r    : CONTENT_TOK <rest> ;
 
CONTENT_TOK    : 'CONTENT/' ;
 
Matching the literal in the parser rule, or creating a TOKEN in the lexer and matching the token in the parser?

Personally I never use literals in the parser and only use the tokens {} section for imaginaries, but there is some argument to say that using literals for punctuation which never goes for other than the parser (the AST does not see it) can be clearer:  

r: X Y Z ';' ;

vs

r: X Y Z SEMI ;

but for me the token name is clearer. It's what you get used to I guess. But, if you search the previous posts on this list you will find many problems caused by people using literals in the parser and not understanding how they are actually lexer tokens, not part of the parser and so on. If you use tokens though (and there is no need to use a suffix like _TOK as it is unnecessary), then you can't really go wrong other than forgetting to translate the grammars in the proper order (and you can't get that wrong if you are targeting Java and use Maven).

Further than this though I always suggest that in production you use separate lexer and parser grammars and use superclasses for any support methods or overrides such as displayRecognitionError. Keep all the code that you can out of the .g file.

Jim

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "il-antlr-interest" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [email protected]
For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

Reply via email to