Tom Lane wrote:
I got interested enough in the psql-with-flex problem to go off and solve it. Attached is a working patch, which I'm now debating whether to apply. Comments solicited...
The patch removes about 200 lines of very spaghetti-ish code in
mainloop.c. However, it adds an 875-line flex source file, which
might be thought a bad tradeoff :-(. One bright spot is that about
half of that total is a direct copy of the main backend lexer, so
it's not really as much new, separately maintainable code as all that.
Also, Andrew Dunstan's patch for supporting dollar-quoting would add
about 100 lines to mainloop.c, versus only a dozen or so lines in the
flex implementation.
Am I missing something, or is dollar quoting not in this patch? Perhaps you have a followup patch?
Once that's taken into account I don't think there is a lot of difference in effective SLOC to maintain. I'm also of the opinion that the new C code in psqlscan.l is much more straightforward than the code removed from mainloop.c, though having just written it, I'm no doubt pretty biased.
Bruce was asking about speed. On normal-size queries I cannot measure any difference at all. For testing purposes I made up a file containing a single 750K query (just a "SELECT big-honking-string-constant", with the string literal broken into lines of 75 bytes). The client-side (psql) CPU time to run this file looks about like this on my machine:
PGCLIENTENCODING UNICODE SJIS
CVS tip 1.57 1.82
flex implementation 0.93 2.33
The flex implementation is consistently faster than CVS tip when dealing with backend-compatible encodings (such as UTF-8). It's consistently slower when it has to deal with a non-backend-safe encoding such as SJIS or Big5. But for real-world cases the differential is down in the noise either way.
I'm inclined to apply this but I can see where a person not comfortable
with flex might feel differently. Opinions?
In principle this is the Right Thing (tm).
We use flex elsewhere, of course, so the fact that it is a flex scanner doesn't seem to me to be any sort of strong argument.
If we are going to do this we need to get it in ASAP, so it gets plenty of beating on.
cheers
andrew
---------------------------(end of broadcast)--------------------------- TIP 4: Don't 'kill -9' the postmaster
