Jim, I have a question regarding your comment on case insensitivity. I have been using the "slowest" case insensitive lexer technique, as this is the first I have seen a viable alternative (on the page that you linked to). The grammar I am working with is a bit strange in that all of the keywords in the language are case insensitive, but some rules, such as variable names, are case sensitive. My question is, how far reaching is the setUcaseLA() function (I am using the C target)? My variable name rule accepts both uppercase and lowercase letters, and when I do $tok.text->chars, I need to get the string in the original case that was entered. So long as that is unaffected, I will be happy to get rid of all of my "fragment A : ('A'|'a');" rules.
Thanks, - Justin On 3/22/2011 5:27 PM, Jim Idle wrote: >> -----Original Message----- >> From: Ruslan Zasukhin [mailto:ruslan_zasuk...@valentina-db.com] >> Sent: Tuesday, March 22, 2011 2:21 PM > >>> However, using lower case literals in your parser directly is not a >>> good idea. Use real tokens so that you error messages are better >> Simple example, please? > Instead of: > > rule : 'join' somerule; > > Use: > > rule : JOIN somerule; > > // Lexer rule to match: > // > JOIN : 'join'; > > And for case insensitivity I specify the token specs all in UPPPER rather > than lower and then override the input stream as per: > > http://www.antlr.org/wiki/pages/viewpage.action?pageId=1782 > > Although someone has added instructions for generating the slowest case > insensitive lexers in the world with individual letter rules. Use the > input stream override method in general. > > > >> >>> and remember >>> that SQL is generally case insensitive so you will need a [trivial] >>> custom input stream. >> Of course we do remember this :) >> >> And after grammar start to breath, we will yet work on >> * case-insensitive of SQL text >> * UTF-16 for input -- clarify .. > > UTF-16 input encoding is just a matter of telling the Java input stream to > open the file in that encoding. > > Jim > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: > http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.