Re: [Python-Dev] Move encoding_decl to the top of Grammar/Grammar?

2008-12-02 Thread Thomas Lee
Here's the corresponding tracker issue: http://bugs.python.org/issue4347 I've uploaded a patch there anyway, since I'm going to need this stuff working for a presentation I'm giving tomorrow. Cheers, T Thomas Lee wrote: Hi all, Currently, Parser/parsetok.c has a depe

[Python-Dev] Move encoding_decl to the top of Grammar/Grammar?

2008-12-01 Thread Thomas Lee
Hi all, Currently, Parser/parsetok.c has a dependency on graminit.h. This can cause headaches when rebuilding after adding new syntax to Grammar/Grammar because parsetok.c is part of pgen, which is responsible for *generating* graminit.h. This circular dependency can result in parsetok.c us

Re: [Python-Dev] when is path==NULL?

2008-09-30 Thread Thomas Lee
Ulrich Eckhardt wrote: Hi! I'm looking at trunk/Python/sysmodule.c, function PySys_SetArgv(). In that function, there is code like this: PyObject* path = PySys_GetObject("path"); ... if (path != NULL) { ... } My intuition says that if path==NULL, something is very wrong. At least

Re: [Python-Dev] when is path==NULL?

2008-09-30 Thread Thomas Lee
Ulrich Eckhardt wrote: Hi! I'm looking at trunk/Python/sysmodule.c, function PySys_SetArgv(). In that function, there is code like this: PyObject* path = PySys_GetObject("path"); ... if (path != NULL) { ... } My intuition says that if path==NULL, something is very wrong. At least

Re: [Python-Dev] lnotab and the AST optimizer

2008-07-24 Thread Thomas Lee
Antoine Pitrou wrote: Thomas Lee vector-seven.com> writes: By the way, you were right about JUMP_IF_TRUE/JUMP_IF_FALSE. It's far too late. Apologies. I'm still pretty sure this is the peepholer's doing, Yes indeed. Which is what's being achieved with

Re: [Python-Dev] lnotab and the AST optimizer

2008-07-24 Thread Thomas Lee
Antoine Pitrou wrote: Antoine Pitrou pitrou.net> writes: In http://bugs.python.org/issue2459 ("speedup for / while / if with better bytecode") I had the same problem and decided to change the lnotab format so that line number increments are signed bytes rather than unsigned. By the wa

Re: [Python-Dev] lnotab and the AST optimizer

2008-07-24 Thread Thomas Lee
Antoine Pitrou wrote: Hi, Hi. Thanks for getting back to me so quickly. I can even respond before I have to drag myself off to bed. :) I'm making some good progress with the AST optimizer, and now the main thing standing in my way is lnotab. Currently lnotab expects bytecode sequencing t

[Python-Dev] lnotab and the AST optimizer

2008-07-24 Thread Thomas Lee
I'm making some good progress with the AST optimizer, and now the main thing standing in my way is lnotab. Currently lnotab expects bytecode sequencing to be roughly in-sync with the order of the source file and a few things that the optimizer does (e.g. swapping the bodies of an if/else after

Re: [Python-Dev] Python VM

2008-07-21 Thread Thomas Lee
Martin v. Löwis wrote: Jakob, This looks fairly correct. A few comments below. Control Flow The calling sequence is: main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) -> run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx() (ce

Re: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None)

2008-06-15 Thread Thomas Lee
Georg Brandl wrote: We can of course decide to make assignment to True and False illegal in 2.7 :) Georg Great to know that's an option. There's little-to-no chance of this making 2.6. I might just avoid trying to treat True/False as "real" constants until there's been a proper discussio

Re: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None)

2008-06-15 Thread Thomas Lee
Georg Brandl wrote: Remember that it must still be possible to write (in 2.6) True = 0 assert not True Ah of course. Looks like I should just avoid optimizations of Name("True") and Name("False") all together. That's a shame! Cheers, T Georg Thomas Lee schrieb

Re: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None)

2008-06-15 Thread Thomas Lee
Benjamin Peterson wrote: On Sun, Jun 15, 2008 at 8:11 AM, Thomas Lee <[EMAIL PROTECTED]> wrote: The simplest options I can think of to remedy this: 1. A setattr hack: setattr(__import__(__name__), "True", True) 2. Remove all optimization of Name("True") and

Re: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None)

2008-06-15 Thread Thomas Lee
Option 4 just struck me: only optimize Name nodes if they have a Load ctx. This makes even more sense: in a Store context, we almost invariably want the name rather than the constant. Cheers, T Thomas Lee wrote: My work on the AST optimizer has led me down the path of attempting to replace

[Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None)

2008-06-15 Thread Thomas Lee
ards #3 at the moment as it seems like it's going to be the cleanest approach and makes a lot of sense -- at least on the surface. Can anybody think of problems with this approach? Cheers, T Thomas Lee wrote: Martin v. Löwis wrote: The question is, what is the specification for Pyt

Re: [Python-Dev] Assignment to None

2008-06-10 Thread Thomas Lee
Martin v. Löwis wrote: The question is, what is the specification for Python. Now, that's a more interesting question than the question originally asked (which I interpreted as "why does it work the way it works"). The only indication in the specification of that feature I could find was:

Re: [Python-Dev] Assignment to None

2008-06-09 Thread Thomas Lee
Tony Nelson wrote: At 4:46 PM +0100 6/9/08, Michael Foord wrote: Alex Martelli wrote: The problem is more general: what if a member (of some external object we're proxying one way or another) is named print (in Python < 3), or class, or...? To allow foo.print or bar.class would requir

Re: [Python-Dev] Tuple pack/unpack and the definition of AST Assign nodes

2008-06-09 Thread Thomas Lee
Nick Coghlan wrote: I haven't looked at that code recently, but I believe the ADSL sequence in the assignment node is for statements where there are actually multiple assignment targets, such as: >>> p = x, y = 1, 2 >>> p, x, y ((1, 2), 1, 2) Cheers, Nick. Ah I see. A quick test verifies

[Python-Dev] Tuple pack/unpack and the definition of AST Assign nodes

2008-06-09 Thread Thomas Lee
In porting one of the old peephole optimizations to the new AST compiler I noticed something weird going on with the following code: a, b, c = 1, 2, 3 Now, as you would expect this gets parsed into an Assign node. That Assign node looks like the following: Assign.targets = [Tuple(Name(a), Na

[Python-Dev] availability of httplib.HTTPResponse.close

2008-05-14 Thread Thomas Lee
I was debating whether this was truly a question for python-dev or if I should take it to one of the user lists. Ultimately it feels like a question about the implementation of a core module, so hopefully nobody minds me posting it here. :) Although not listed as a public API method in the doc

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Thomas Lee
Nick Coghlan wrote: Steve Holden wrote: While not strictly related to the global statement, perhaps Adam refers to the possibility of optimizing away code with an assignment which would make a name be recognized as local? If you're worried about "yield" disappearing you should also be worried

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Thomas Lee
Adam Olsen wrote: On Thu, May 8, 2008 at 5:22 PM, Thomas Lee <[EMAIL PROTECTED]> wrote: Nick Coghlan wrote: There are a lot of micro-optimisations that are actually context independent, so moving them before the symtable pass should be quite feasible - e.g. replacing "return

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Thomas Lee
Nick Coghlan wrote: There are a lot of micro-optimisations that are actually context independent, so moving them before the symtable pass should be quite feasible - e.g. replacing "return None" with "return", stripping dead code after a return statement, changing a "if not" statement into an

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-07 Thread Thomas Lee
Nick Coghlan wrote: As Thomas mentions in a later message, making it possible to annotate nodes would permit Functions to be annotated as being a generator at the AST stage (currently it is left to the bytecode compiler's symtable generation pass to make that determination). Although I gues

Re: [Python-Dev] Module Suggestion: ast

2008-05-04 Thread Thomas Lee
I'm in the process of writing C code for the purposes of traversing AST nodes in the AST optimization branch. This seems to be an ideal case for code generation based on the ASDL representation of the AST as we're currently doing for Python-ast.[ch]. I'm already considering this approach for s

[Python-Dev] AST Optimization: Branch Elimination in Generator Functions

2008-05-02 Thread Thomas Lee
The next problem that cropped up during the implementation of the AST code optimizer is related to branch elimination and the elimination of any code after a return. Within a FunctionDef node, we would (ideally) like to blow away If nodes with a constant - but false - test expression. e.g.:

[Python-Dev] AST Optimization: Branch Elimination in Generator Functions

2008-05-02 Thread Thomas Lee
The next problem that cropped up during the implementation of the AST code optimizer is related to branch elimination and the elimination of any code after a return. Within a FunctionDef node, we would (ideally) like to blow away If nodes with a constant - but false - test expression. e.g.:

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-02 Thread Thomas Lee
Martin v. Löwis wrote: This leaves us with a few options: 5. Reuse/Abuse Num(object) for arbitrary constants. AFAICT, this should work out of the box. Eek. It *does* seem like Num would work out of the box, but would this be a good idea? What about *replacing* Num with Const? Mig

[Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-04-30 Thread Thomas Lee
Hi all, I've been working on optimization of the AST, including the porting of the old bytecode-level optimizations to the AST level. A few questions have come up in the process of doing this, all of which are probably appropriate for discussion on this list. The code I'm referring to here ca

Re: [Python-Dev] Module Suggestion: ast

2008-04-29 Thread Thomas Lee
Just a thought, but it would be great if this could be implemented over the top of a C layer that operates on real AST nodes (rather than the PyObject representation of those nodes). I'm working on stuff to perform code optimization at the AST level (see the tlee-ast-optimize branch), and the f

Re: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and June 21st-22nd.

2008-04-22 Thread Thomas Lee
Anyone in Melbourne, Australia keen for the first sprint? I'm not sure if I'll be available, but if I can it'd be great to work with some others. Failing that, it's red bull and pizza in my lounge room :) I've been working on some neat code for an AST optimizer. If I'm free that weekend, I'll

Re: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and June 21st-22nd.

2008-04-17 Thread Thomas Lee
Anybody in Melbourne keen for this? Not sure if I'll be able to make it myself, but I'd be interested to know if there's anybody in the area keen to do the sprint. Cheers, T Tarek Ziadé wrote: > On Wed, Apr 16, 2008 at 8:40 PM, Michael Foord > <[EMAIL PROTECTED]> wrote: > >> Trent Nelson wro

Re: [Python-Dev] [python] Should we do away with unbound methods in Py3k?

2007-11-24 Thread Thomas Lee
Michael Foord wrote: > Guido van Rossum wrote: > >> I'm asking a Py3k question on python-dev because I'd like to have >> opinions from people who haven't thought about Py3k much yet. Consider >> the following example: >> >> class C: >> def foo(self): pass >> >> C.foo(42) >> >> This cur

[Python-Dev] zipfile.ZipFile('foo.zip', 'a'): file not found -> create?

2006-07-06 Thread Thomas Lee
Hi all, In reference to: http://sourceforge.net/tracker/index.php?func=detail&aid=1514451&group_id=5470&atid=105470 I wrote a patch for this "bug", but a valid point was raised by Ronald Oussorren: this borders on being more of a "feature" than a bug fix, although - IMHO - this fix improves c

Re: [Python-Dev] Switch statement

2006-06-18 Thread Thomas Lee
On Sat, Jun 10, 2006 at 05:53:14PM -0500, [EMAIL PROTECTED] wrote: > > Thomas> As the subject of this e-mail says, the attached patch adds a > Thomas> "switch" statement to the Python language. > > Thanks for the contribution. I patched my sandbox and it built just fine. > I'm going out

Re: [Python-Dev] Switch statement

2006-06-12 Thread Thomas Lee
On Mon, Jun 12, 2006 at 11:33:49PM +0200, Michael Walter wrote: > Maybe "switch" became a keyword with the patch.. > > Regards, > Michael > That's correct. > On 6/12/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote: > > > > Could you upload your patch to SourceForge ? Then I could add > > it to the

Re: [Python-Dev] Switch statement

2006-06-10 Thread Thomas Lee
On Sat, Jun 10, 2006 at 05:53:14PM -0500, [EMAIL PROTECTED] wrote: > * Aside from the modified Grammar file there is no documentation. > * There are no test cases. > * Can you submit a patch on SourceForge? All have been addressed, although I'm not sure if I've covered everywhere I nee

[Python-Dev] New Future Keywords

2006-03-05 Thread Thomas Lee
Hi, Just a quick question: how can I add new future keywords to Python? I need to add a new (Python) keyword to the language, but there seems to be a few different source files that I need to modify. Just want to make sure I'm doing it the right way before I go unleashing a nasty broken patch

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-30 Thread Thomas Lee
Nick Coghlan wrote: >Greg Ewing wrote: > > >>Neal Norwitz wrote: >> >> >> >>>I'm mostly convinced that using PyObjects would be a good thing. >>>However, making the change isn't free as all the types need to be >>>created and this is likely quite a bit of code. >>> >>> >>Since they're

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-23 Thread Thomas Lee
Neil Schemenauer wrote: >Fredrik Lundh <[EMAIL PROTECTED]> wrote: > > >>Thomas Lee wrote: >> >> >> >>>Even if it meant we had just one function call - one, safe function call >>>that deallocated all the memory allocated within a func

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-17 Thread Thomas Lee
Portability may also be an issue to take into consideration: http://www.eskimo.com/~scs/C-faq/q7.32.html http://archives.neohapsis.com/archives/postfix/2001-05/1305.html Cheers, Tom Alex Martelli wrote: >On Nov 17, 2005, at 12:46 PM, Brett Cannon wrote: >... > > >>>alloca? >>> >>>(duck) >

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
NULL); assert(finally != NULL); result_st = TryFinally(pool, body, finally, LINENO(n)); if (result_st == NULL) AST_FAILURE(pool, NULL); } /* pool deallocated when c->pool is deallocated */ return AST_SUCCESS(pool, result_st); } Nick Coghlan wrote: >Thoma

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
By the way, I liked the sound of the arena/pool tree - really good idea. Thomas Lee wrote: >Niko Matsakis wrote: > > > >>>Boy am I wanting RAII from C++ for automatic freeing when scope is >>>left. Maybe we need to come up with a similar thing, like all memory

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
Niko Matsakis wrote: >>Boy am I wanting RAII from C++ for automatic freeing when scope is >>left. Maybe we need to come up with a similar thing, like all memory >>that should be freed once a scope is left must use some special struct >>that stores references to all created memory locally and then

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
As the writer of the crappy code that sparked this conversation, I feel I should say something :) Brett Cannon wrote: >On 11/15/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > >>On 11/15/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote: >> >> >>>Thanks for the message. I was going to suggest t

[Python-Dev] PEP 341 patch & memory management (was: Memory management in the AST parser & compiler)

2005-11-15 Thread Thomas Lee
Interesting trick! The PEP 341 patch is now using Marek's 'do ... while' resource cleanup trick instead of the nasty goto voodoo. I've also fixed the last remaining bug that Neal pointed out. I'm running the unit tests right now, shall have the updated (and hopefully final) PEP 341 patch up on

[Python-Dev] Implementation of PEP 341

2005-11-13 Thread Thomas Lee
Hi all, I've been using Python for a few years and, as of a few days ago, finally decided to put the effort into contributing code back to the project. I'm attempting to implement PEP 341 (unification of try/except and try/finally) against HEAD. However, this being my first attempt at a chang