https://issues.dlang.org/show_bug.cgi?id=14679
Bruno Medeiros <bruno.do.medeiros+deeb...@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bruno.do.medeiros+deebugz@g | |mail.com --- Comment #4 from Bruno Medeiros <bruno.do.medeiros+deeb...@gmail.com> --- (In reply to Andrei Alexandrescu from comment #3) > Yah, the alternate grammar needs to recognize comments, string literals, and > newlines (for line counting purposes). > > One nice thing about not doing tokenization is a lot of memory is saved. Why does tokenization consume memory? I imagine you can just have the token point to a slice of the source? --