Re: decorators and closures
On Monday, November 21, 2011 10:44:34 PM UTC+8, Andrea Crotti wrote:
> With one colleague I discovered that the decorator code is always
> executed, every time I call
> a nested function:
>
> def dec(fn):
> print("In decorator")
> def _dec():
> fn()
>
> return _dec
>
> def nested():
> @dec
> def fun():
> print("here")
>
> nested()
> nested()
>
> Will give:
> In decorator
> In decorator
>
> So we were wondering, would the interpreter be able to optimize this
> somehow?
> I was betting it's not possible, but I'm I would like to be wrong :)
I love to use decorators. I did the same things to functions and structures in
c long time ago. I think that might be the dark night era in programming in
the early 90's long long ago.
Cheers.
--
http://mail.python.org/mailman/listinfo/python-list
Re: decorators and closures
On Mon, 21 Nov 2011 14:44:34 +, Andrea Crotti wrote: > With one colleague I discovered that the decorator code is always > executed, every time I call a nested function: "def" is a statement which is executed at runtime. Often people will talk about "definition time" instead of "compile time". Python compiles your source into byte code (compile time), then executes the byte code. The function doesn't actually get created until the byte code is executed, i.e. at run time. This is not as slow as it sounds, because the function is created from pre-compiled parts. In effect, if you have a module: x = 23 def spam(a): print x print x+1 return x**3 then the body of the function is compiled into a "code object" at compile time, and at runtime the function object itself is assembled from the code object, name, and whatever other bits and pieces are needed. In pseudo-code, the byte code looks like this: bind name "x" to object 23 build a function object "spam" from code object bind name "spam" to function object The hard part is creating the code object, as that requires parsing the source code of the body and generating byte code. That's done once, ahead of time, so the actual "build a function" part is fast. Now, if you have an ordinary nested function: def spam(a): def ham(b): return a+b return ham(a+42) # returns a numeric value or a closure: def spam(a): def ham(b): return a+b return ham # returns a closure (function object) the process is no different: the inner function doesn't get created until runtime, that is, when spam gets *called*. But it gets created from parts that were prepared earlier at compile time, and so is fast. Add a decorator, and the basic process remains. Remember that decorator syntax is just syntactic sugar. This: @decorator def spam(): pass is exactly the same as this: def spam(): pass spam = decorator(spam) which clearly has to be done at runtime, not compile time. That applies regardless of whether the function is nested or top-level. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: decorators and closures
On 11/21/2011 05:11 PM, Dave Angel wrote:
You didn't mention what version of Python you're running. With Python
2, I got very different results. So I switched to Python 3.2, and I
still don't get exactly what you have.
A closure is needed if there's some non-global data outside the
function definition (code object) that's needed by the function
object. As you supply the code I don't need a closure. But if I add
a local variable in test_decorate(), and refer to it in dec(), then I
get one. Not the same as yours.
You left out the import and the definition line for test_decorate.
Did you leave anything else? And what version of Python are you
using? Are you perhaps running in a shell, as opposed to running code
directly from a source file?
I use python 2.7, and actually the whole source is this (test_decorate.py):
def dec(fn):
def _dec():
fn()
return _dec
@dec
def fun():
print("here")
fun()
Using ipython:
import test_decorate
dis.dis(test_decorate)
--
http://mail.python.org/mailman/listinfo/python-list
Re: decorators and closures
On 11/21/2011 10:35 AM, Andrea Crotti wrote:
On 11/21/2011 03:06 PM, Dave Angel wrote:
Your function 'nested' isn't nested, 'fun' is. What you discovered is
that a decorator is always executed, every time a nested decorated
function is defined.
You've also ust proved that it would be an incompatible change.
Doesn't that answer the question? An optimizer that changes the
behavior isn't usually desirable.
Yes sure I think it makes perfectly sense, because you actually redefine
a local variable every time..
Another thing (which was also the reason of the subject), I tried to
disassemble the following:
def dec(fn):
def _dec():
fn()
return _dec
@dec
def fun():
print("here")
fun()
And I get this:
In [29]: dis.dis(test_decorate)
Disassembly of dec:
2 0 LOAD_CLOSURE 0 (fn)
3 BUILD_TUPLE 1
6 LOAD_CONST 1 ()
9 MAKE_CLOSURE 0
12 STORE_FAST 1 (_dec)
5 15 LOAD_FAST 1 (_dec)
18 RETURN_VALUE
Disassembly of fun:
3 0 LOAD_DEREF 0 (fn)
3 CALL_FUNCTION 0
6 POP_TOP
7 LOAD_CONST 0 (None)
10 RETURN_VALUE
Looking up the definition of the single calls didn't help much, so why
do we need
for example MAKE_CLOSURE?
Is MAKE_CLOSURE just more generic maybe?
You didn't mention what version of Python you're running. With Python
2, I got very different results. So I switched to Python 3.2, and I
still don't get exactly what you have.
A closure is needed if there's some non-global data outside the function
definition (code object) that's needed by the function object. As you
supply the code I don't need a closure. But if I add a local variable
in test_decorate(), and refer to it in dec(), then I get one. Not the
same as yours.
You left out the import and the definition line for test_decorate. Did
you leave anything else? And what version of Python are you using? Are
you perhaps running in a shell, as opposed to running code directly from
a source file?
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
Re: decorators and closures
On 11/21/2011 03:06 PM, Dave Angel wrote:
Your function 'nested' isn't nested, 'fun' is. What you discovered is
that a decorator is always executed, every time a nested decorated
function is defined.
You've also ust proved that it would be an incompatible change.
Doesn't that answer the question? An optimizer that changes the
behavior isn't usually desirable.
Yes sure I think it makes perfectly sense, because you actually redefine
a local variable every time..
Another thing (which was also the reason of the subject), I tried to
disassemble the following:
def dec(fn):
def _dec():
fn()
return _dec
@dec
def fun():
print("here")
fun()
And I get this:
In [29]: dis.dis(test_decorate)
Disassembly of dec:
2 0 LOAD_CLOSURE 0 (fn)
3 BUILD_TUPLE 1
6 LOAD_CONST 1 (0x1c4b930, file "test_decorate.py", line 2>)
9 MAKE_CLOSURE 0
12 STORE_FAST 1 (_dec)
5 15 LOAD_FAST1 (_dec)
18 RETURN_VALUE
Disassembly of fun:
3 0 LOAD_DEREF 0 (fn)
3 CALL_FUNCTION0
6 POP_TOP
7 LOAD_CONST 0 (None)
10 RETURN_VALUE
Looking up the definition of the single calls didn't help much, so why
do we need
for example MAKE_CLOSURE?
Is MAKE_CLOSURE just more generic maybe?
--
http://mail.python.org/mailman/listinfo/python-list
Re: decorators and closures
On 11/21/2011 09:44 AM, Andrea Crotti wrote:
With one colleague I discovered that the decorator code is always
executed, every time I call
a nested function:
def dec(fn):
print("In decorator")
def _dec():
fn()
return _dec
def nested():
@dec
def fun():
print("here")
nested()
nested()
Will give:
In decorator
In decorator
So we were wondering, would the interpreter be able to optimize this
somehow?
I was betting it's not possible, but I'm I would like to be wrong :)
Your function 'nested' isn't nested, 'fun' is. What you discovered is
that a decorator is always executed, every time a nested decorated
function is defined.
You've also ust proved that it would be an incompatible change. Doesn't
that answer the question? An optimizer that changes the behavior isn't
usually desirable.
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
