John Snow <js...@redhat.com> writes: > On 1/23/20 2:19 AM, Markus Armbruster wrote: [...] >> Any conventional parser can be made streaming by turning it into a >> coroutine. This is probably the simplest solution for handwritten >> streaming LL parsers, because it permits recursive descent. In Python, >> I'd try a generator. >> >> Our actual solution for QMP predates coroutine support in QEMU, and is >> rather hamfisted: >> >> * Streaming lexer: it gets fed characters one at a time, and when its >> state machine says "token complete", it feeds the token to the >> "streamer". >> >> * "Streamer": gets fed tokens one at a time, buffers them up counting >> curly and square bracket nesting until the nesting is zero, then >> passes the buffered tokens to the parser. >> >> * Non-streaming parser: it gets fed a sequence of tokens that constitute >> a full expression. >> >> The best I can say about this is that it works. The streamer's token >> buffer eats a lot of memory compared to a real streaming parser, but in >> practice, it's a drop in the bucket. >> > > I looked into this at one point. I forget why I didn't like it. I had
Because it's confusing? Took me a while to get used to it. > some notion that I should replace this one too, but forget exactly why. > Maybe it wasn't that bad, if I've forgotten. Bigger fish to fry. [...]