On Fri, 18 Mar 2005, Tom Lane wrote:
Marco Colombo <[EMAIL PROTECTED]> writes:Right now I'm parsing the string first, changing the resulting parse tree adding missing nodes (def, INDENT, DEINDENT) and then compiling it.
Hmmm ... is this really going to be simpler or more robust than lexing the string carefully enough to insert tabs at the right places? The impression I had so far was that you'd only need to understand about Python's string-literal conventions to get that right ... and that's something that's not likely to change. I'm not so sure that parse trees can be regarded as an immutable API.
regards, tom lane
I've completed a proof of concept, I think I can answer:
- simpler? not at all. It requires understanding of how the parser works. The whole thing is about 50 lines long, but quite a bit of parser magic is going on. And I'm far from the point I can be confident about it doing always the right thing. I still have to handle (de)allocations correctly.
- more robust - yes. The only way to make sure we're lexing the string the same way python does is to use its lexer. Every single difference however subtle would call for a bug. And it's re-invening the wheel. But there's no way you can work at lexer level that I'm aware of. That is, to add tokens before sending them to the parser. So you have to work on the parser output.
- I have no idea if the "node" API is immutable at all. For sure, the interface I'm using is one or two levels below the current one, and yes, it's more likely to change. I share your concerns here.
If our problem is only string literals, maybe we can handle them with a dedicated lexer. Python string literals are quite complex (compared to other languages):
http://docs.python.org/ref/strings.html
but not that hard.
Still, my first concern is that one day we find another corner case in python syntax that makes our "blind" tab adding fail. And we're back to square one.
BTW, I'm not preparing a patch for now, I'm working with a test program. As soon as I finish it, either I'll post it or prepare a patch against plpython.c, for consideration. I won't say it is ready for inclusion until someone else more knowledgeable than me on both PostgreSQL and python embedding looks at it, anyway.
.TM.
--
____/ ____/ /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_____/ _____/ _/ [EMAIL PROTECTED]---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives?
http://archives.postgresql.org
