Re: [Python-Dev] weakref enhancements

[tomer filiba]

this may still be premature, but i see people misunderstood the purpose.

weakattrs are not likely to be used "externally", out of the scope of
the object.
they are just meant to provide an easy to use means for not holding cyclic
references between parents and children.

many graph-like structures, i.e., rpyc's node and proxies, are interconnected
in both ways, and weakattrs help to solve that: i don't want a proxy of a node
to keep the node alive.

weakmethods are used very similarly. nodes have a method called
"getmodule", that performs remote importing of modules. i expose these
modules as a namespace object, so you could do:
>>> mynode.modules.sys
or
>>> mynode.modules.xml.dom.minidom.parseString
instead of
>>> mynode.getmodule("xml.dom.minidom").parseString

here's a sketch:

def ModuleNamespace:
def __init__(self, importerfunc):
self.importerfunc = importerfunc

class Node:
def __init__(self, stream):

self.modules = ModuleNamespace(self.getmodule)

@ weakmethod
def getmodule(self, name):
 

i define this getmodule method as a *weakmethod*, so the mere existence
of the ModuleNamespace instance will not keep the node alive. when the
node loses all external references, the ModuleNamespace should just
"commit suicide", and allow the node to be reclaimed.

yes, you can do all of these with today's weakref, but it takes quite
a lot of hassle to manually set up weakproxies every time.


-tomer

On 9/29/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> [Alex Martelli]
>
> >I've had use cases for "weakrefs to boundmethods" (and there IS a
> >Cookbook recipe for them),
> >
> Weakmethods make some sense (though they raise the question of why bound
> methods are being kept when the underlying object is no longer in use --
> possibly as unintended side-effect of aggressive optimization).
>
> I'm more concerned about weakattr which hides the weak referencing from
> client code when it is usually the client that needs to know about the
> refcounts:
>
>n = SomeClass(x)
>obj.a = n
>del n  # hmm, what happens now?
>
> If obj.a is a weakattr, then n get vaporized; otherwise, it lives.
>
> It is clearer and less error-prone to keep the responsibility with the
> caller:
>
>n = SomeClass(x)
>obj.a = weakref.proxy(n)
>del n # now, it is clear what happens
>
> The wiki-space example shows objects that directly assign a copy of self
> to an attribute of self. Even in that simplified, self-referential
> example, it is clear that correct functioning (when __del__ gets called)
> depends knowing whether or not assignments are creating references.
> Accordingly, the code would be better-off if the weak-referencing
> assignment was made explicit rather than hiding the weak-referencing
> wrapper in a descriptor.
>
>
>
> Raymond
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Greg Ewing]

Raymond Hettinger wrote:
> Also, I question the utility of maintaining a weakref to a method or 
> attribute instead of holding one for the object or class.  As long as 
> the enclosing object or class lives, so too will their methods and 
> attributes.  So what is the point of a tighter weakref granualarity?

I think you're misunderstanding what the OP means. A
weak attribute isn't a weak reference to an attribute,
it's an attribute that holds a weak reference and is
automatically dereferenced when you access it.

A frequent potential use case is parent-child relationships.
To avoid creating cycles you'd like to make the link
from child to parent weak, but doing that with raw
weakrefs is somewhat tedious and doesn't feel worth
the bother. If I could just declare the attribute
to be weak and then use it like a normal attribute
from then on, I would probably use this technique
more often.

> So, before being entertained for addition to the standard library, this 
> idea should probably first be posted as an ASPN recipe,

That's a reasonable idea.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Bob Ippolito]

On 9/28/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> > There are *definitely* use cases for keeping bound methods around.
> >
> > Contrived example:
> >
> >one_of = set([1,2,3,4]).__contains__
> >filter(one_of, [2,4,6,8,10])
>
> ISTM, the example shows the (undisputed) utility of regular bound methods.
>
> How does it show the need for methods bound weakly to the underlying object,
> where the underlying can be deleted while the bound method persists, alive but
> unusable?

It doesn't. I seem to have misinterpreted your "Weakmethods have some
use (...)" sentence. Sorry for the noise.

-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Raymond Hettinger]

> There are *definitely* use cases for keeping bound methods around.
>
> Contrived example:
>
>one_of = set([1,2,3,4]).__contains__
>filter(one_of, [2,4,6,8,10])

ISTM, the example shows the (undisputed) utility of regular bound methods.

How does it show the need for methods bound weakly to the underlying object,
where the underlying can be deleted while the bound method persists, alive but 
unusable?


Raymond 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Bob Ippolito]

On 9/28/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> [Alex Martelli]
>
> >I've had use cases for "weakrefs to boundmethods" (and there IS a
> >Cookbook recipe for them),
> >
> Weakmethods make some sense (though they raise the question of why bound
> methods are being kept when the underlying object is no longer in use --
> possibly as unintended side-effect of aggressive optimization).

There are *definitely* use cases for keeping bound methods around.

Contrived example:

one_of = set([1,2,3,4]).__contains__
filter(one_of, [2,4,6,8,10])

-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Raymond Hettinger]

[Alex Martelli]

>I've had use cases for "weakrefs to boundmethods" (and there IS a
>Cookbook recipe for them),
>
Weakmethods make some sense (though they raise the question of why bound 
methods are being kept when the underlying object is no longer in use -- 
possibly as unintended side-effect of aggressive optimization).

I'm more concerned about weakattr which hides the weak referencing from 
client code when it is usually the client that needs to know about the 
refcounts:

   n = SomeClass(x)
   obj.a = n
   del n  # hmm, what happens now?

If obj.a is a weakattr, then n get vaporized; otherwise, it lives.

It is clearer and less error-prone to keep the responsibility with the 
caller:

   n = SomeClass(x)
   obj.a = weakref.proxy(n)
   del n # now, it is clear what happens

The wiki-space example shows objects that directly assign a copy of self 
to an attribute of self. Even in that simplified, self-referential 
example, it is clear that correct functioning (when __del__ gets called) 
depends knowing whether or not assignments are creating references.  
Accordingly, the code would be better-off if the weak-referencing 
assignment was made explicit rather than hiding the weak-referencing 
wrapper in a descriptor.



Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Alex Martelli]

On 9/28/06, tomer filiba <[EMAIL PROTECTED]> wrote:
> > I'm sceptical that these would find use in practice.
> > [..]
> > Also, I question the utility of maintaining a weakref to a method or
> > attribute instead of holding one for the object or class.  As long as
> > the enclosing object or class lives, so too will their methods and
> > attributes.  So what is the point of a tighter weakref granualarity?
>
> i didn't just came up with them "out of boredom", i have had specific
> use cases for these, mainly in rpyc3000... but since the rpyc300
> code base is still far from completion, i don't want to give examples
> at this early stage.
>
> however, these two are theoretically useful, so i refactored them out
> of my code into recipes.

I've had use cases for "weakrefs to boundmethods" (and there IS a
Cookbook recipe for them), as follows: sometimes I'm maintaining a
container of callables, which may be of various kinds including
functions, boundmethods, etc; but I'd like the mere presence of a
callable in the container not to keep the callable alive (especially
when the callable in turn keeps alive an object with possibly massive
state). In practice I use wrapping and tricks, but it would be nice to
have cleaner standard library support for this. (Often the container
needs to be some form of a Queue.Queue, since queues of callables are
a form I use very often to dispatch work requests to worker-threads in
a threadpool).


Alex
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[tomer filiba]

> I'm sceptical that these would find use in practice.> [..]> Also, I question the utility of maintaining a weakref to a method or> attribute instead of holding one for the object or class.  As long as
> the enclosing object or class lives, so too will their methods and> attributes.  So what is the point of a tighter weakref granualarity?i didn't just came up with them "out of boredom", i have had specific 
use cases for these, mainly in rpyc3000... but since the rpyc300 code base is still far from completion, i don't want to give examples at this early stage.however, these two are theoretically useful, so i refactored them out
of my code into recipes.-tomerOn 9/28/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> Also, I question the utility of maintaining a weakref to a method or> attribute instead of holding one for the object or class.Strike that paragraph -- the proposed weakattrs have references away from the
object, not to the object.Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Raymond Hettinger]

> Also, I question the utility of maintaining a weakref to a method or
> attribute instead of holding one for the object or class.

Strike that paragraph -- the proposed weakattrs have references away from the 
object, not to the object.


Raymond 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] weakref enhancements

[Raymond Hettinger]

tomer filiba wrote:

> i'd like to suggest adding weak attributes and weak methods to the std 
> weakref
> module.

 . . .

>
> i think these two features are quite useful, and being part of the 
> stdlib, would
> provide programmers with easy-to-use solutions to object-aliveness issues.
>
> more info, examples, and suggested implementation:
> * http://sebulba.wikispaces.com/recipe+weakattr
> * http://sebulba.wikispaces.com/recipe+weakmethod 
> 
>

I'm sceptical that these would find use in practice.  The cited links 
have only toy examples and as motivation reference Greg Ewing's posting 
saying only "I'm thinking it would be nice . . . This could probably be 
done fairly easily with a property descriptor." 

Also, I question the utility of maintaining a weakref to a method or 
attribute instead of holding one for the object or class.  As long as 
the enclosing object or class lives, so too will their methods and 
attributes.  So what is the point of a tighter weakref granualarity?

So, before being entertained for addition to the standard library, this 
idea should probably first be posted as an ASPN recipe, then we can see 
if any use cases emerge in actual practice.  Then we could look at 
sample code fragments to see if any real-world code is actually improved 
with the new toys.  My bet is that very few will emerge, that most would 
be better served by a simple decorator, and that an expanding weakref 
zoo will only making the module more difficult to learn.


Raymond






___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] weakref enhancements

[tomer filiba]

i'd like to suggest adding weak attributes and weak methods to the std weakrefmodule. weakattrs are weakly-referenced attributes. when the value they reference is no longer strongly-referenced by something else, the weakattrs "nullify" themselves.
weakmethod is a method decorator, like classmethod et al, that returns "weaklybound" methods. weakmethod's im_self is a weakref.proxy to `self`, which meansthe mere method will not keep the entire instance alive. instead, you'll get a 
ReferenceError.i think these two features are quite useful, and being part of the stdlib, would provide programmers with easy-to-use solutions to object-aliveness issues.more info, examples, and suggested implementation:
* http://sebulba.wikispaces.com/recipe+weakattr* http://sebulba.wikispaces.com/recipe+weakmethod
-tomer
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com