Andrew Dunstan <and...@dunslane.net> writes: > Tom Lane wrote: >> Andrew Dunstan <and...@dunslane.net> writes: >>> I think it would need to be benchmarked. My faint recollection is that >>> the re-entrant lexers are slower. >> >> The flex documentation states in so many words: >> The option `--reentrant' does not affect the performance of the scanner. >> Do you feel a need to verify their claim?
> No, I'll take their word for it. I must have been thinking of something > else. As I got further into this, it turned out that Andrew's instinct was right: it does need to be benchmarked. Although the inner loops of the lexer seem to be the same with or without --reentrant, once you buy into the whole nine yards of --reentrant, --bison-bridge, and a "pure" bison parser, you find out that the lexer's API changes: there are more parameters to yylex() than there used to be. It's also necessary to pass around a yyscanner pointer to all the subroutines in scan.l. (But on the other hand, this eliminates accesses to global variables, which are often not that cheap.) So the "no performance impact" claim isn't telling the whole truth. As best I can tell after some casual testing on a couple of machines, the actual bottom line is that "raw_parser" (ie, the bison and flex processing) is going to be a couple of percent slower with a reentrant grammar and lexer, for typical queries involving a lot of short tokens. Now this disappears into the noise as soon as you include parse analysis (let alone planning and execution), but it is possible to measure the slowdown in a test harness that calls raw_parser only. A possible compromise that I think would avoid most or all of the slowdown is to make the lexer reentrant but not the grammar (so that yylval and yylloc remain as global variables instead of being parameters to yylex). I haven't actually benchmarked that, though. It strikes me as a fairly silly thing to do. If we're going to go for reentrancy I think we should fix both components. I'm willing to live with the small slowdown. Comments? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers