On Mon, Apr 1, 2024 at 4:53 PM Andrew Dunstan <and...@dunslane.net> wrote: > Anyway, here are new patches. I've rolled the new semantic test into the > first patch.
Looks good! I've marked RfC. > json_lex() is not really a very hot piece of code. Sure, but I figure if someone is trying to get the performance of the incremental parser to match the recursive one, so we can eventually replace it, it might get a little warmer. :) > > I think it'd be good for a v1.x of this feature to focus on > > simplification of the code, and hopefully consolidate and/or eliminate > > some of the duplicated parsing work so that the mental model isn't > > quite so big. > > I'm not sure how you think that can be done. I think we'd need to teach the lower levels of the lexer about incremental parsing too, so that we don't have two separate sources of truth about what ends a token. Bonus points if we could keep the parse state across chunks to the extent that we didn't need to restart at the beginning of the token every time. (Our current tools for this are kind of poor, like the restartable state machine in PQconnectPoll. While I'm dreaming, I'd like coroutines.) Now, whether the end result would be more or less maintainable is left as an exercise... Thanks! --Jacob
