On Tuesday, 15 May 2012 at 06:17:31 UTC, Era Scarecrow wrote:
If you don't mind doing some lite informational reading, read up on 'lex and yacc'. ISBN: 1-56592-000-7; Hope I got that right. I'm not telling you to use C (or the other tools mind you), but it does go into handling tokens of multiple types as well as managing them using pointers and unions. A good portion of that can carry over for your lexer and project.
Looks like I can introduce several lex/flex concepts in my lexer, thank you! I've created a task for myself to do this investigation: https://github.com/roman-d-boiko/dct/issues/42
