On Mon, Aug 11, 2014 at 10:50:34PM +0200, Philippe Sigaud via Digitalmars-d-learn wrote: > On Mon, Aug 11, 2014 at 10:09 PM, H. S. Teoh via Digitalmars-d-learn > <digitalmars-d-learn@puremagic.com> wrote: > > On Mon, Aug 11, 2014 at 07:47:44PM +0000, Klaus via Digitalmars-d-learn > > wrote: > >> I mean when writing a D lexer, you necessarly reach the moment when > >> you think: > >> > >> "Oh no! is this feature just here to suck ?" > > > > The crazy variety of ways to write string literals in D, OTOH, *is* > > a bit over the top, as I found out myself when I also tried writing > > a D lexer. :-P > > Out of curiosity, how does a lexer deal with heredocs? It's a sort > of... user-defined token, right?
In Flex, one way you can implement heredocs is to have a separate "mode" where the lexer is scanning for the ending string. So basically you have a sub-lexer that treats the heredoc as three tokens, one that defines the ending string for the heredoc (which is never returned to the caller), one that contains the content of the heredoc, and the terminating token (also never returned to the caller). It's not that much different from any other string literal scanning (the lexer must switch into "string literal mode" where things like \n and \t have a different meaning than in the program code proper, and it exits that mode when it encounters the terminating '"'), except that here, the terminating delimiter is variable. T -- Real men don't take backups. They put their source on a public FTP-server and let the world mirror it. -- Linus Torvalds
