07-Mar-2013 00:52, Timon Gehr пишет:
On 03/05/2013 08:12 PM, Dmitry Olshansky wrote:
...
There is one thing I found a nice abstraction while helping out on D's
lexer in D and I call it mark-slice range. An extension to forward range
it seems.
It's all about buffering and defining a position in input such that you
don't care for anything up to this point. This means that starting from
thusly marked point stuff needs to be kept in buffer, everything prior
to it could be discarded. The 2nd operation "slice" is getting a slice
of some internal buffer from last mark to the current position.
...
The lexer I have built last year does something similar. It allows the
parser to save and restore sorted positions in FIFO order with one
size_t of memory inside the parser's current stack frame (internally,
the lexer only saves the first position). The data is kept in a circular
buffer that grows dynamically in case the required lookahead is too large.
Exactly. Nice to see common patterns resurface, would be good to fit it
elegantly into a native D i/o subsystem.
--
Dmitry Olshansky
