I'll wait for others to jump on this bandwagon... IMO the tempfile
object would be better off not to bother with caching at all...
On Thu, Apr 3, 2008 at 5:33 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Guido van Rossum wrote:
>
> > On Wed, Apr 2, 2008 at 6:30 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> >
> > > One of the issues with porting to Py3k is the problem that __getattr__
> > > and __getattribute__ can't reliably provide special methods like
> __add__
> > > the way __getattr__ could with classic classes. (As first noted by
> Terry
> > > Reedy years ago, and recently seeing some new activity on the bug
> > > tracker [1])
> > >
> > > The culprit here is the fact that __getattribute__ and its associated
> > > machinery is typically never invoked for the methods with dedicated
> tp_*
> > > slots in the C-level type structure.
> > >
> >
> > Well, yes, this is all an intentional part of the new-style class design.
> >
>
> Not complaining, just trying to provide some background for those that may
> not be quite as familiar with the inner workings of typeobject.c :)
>
>
>
> >
> > > What do people think of the idea of providing an extra method on type
> > > objects that goes through all of the C-level special method slots, and
> > > for each one that isn't currently set, does a getattr() on the
> > > associated special name and stores the result (if any) on the current
> > > type object?
> > >
> >
> > Does a getattr on what? Since you seem to be thinking specifically of
> > proxies here, I'm thinking you're doing a getattr on an *instance* --
> > but it seems wrong to base the *type* slots on that.
> >
>
> D'oh, you're right - the specific proxying example I am thinking of (see
> below) does indeed grab bound methods directly from the underlying instance.
> However, I think the idea is salvageable (whether or not it is *worth*
> salvaging is of course a completely different question!).
>
>
>
> >
> > > When converting a proxy class that relies on __getattr__ from classic
> > >
> >
> > Can you show specific code for such a proxy class? I'm having a hard
> > time imagining how it would work (not having used proxies in a really
> > long time...).
> >
>
> From tempfile._TemporaryFileWrapper, which aims to delegate as many
> operations as it can automatically to the underlying file object:
>
> def __getattr__(self, name):
> # Attribute lookups are delegated to the underlying file
> # and cached for non-numeric results
> # (i.e. methods are cached, closed and friends are not)
> file = self.__dict__['file']
> a = getattr(file, name)
> if not issubclass(type(a), type(0)):
> setattr(self, name, a)
> return a
>
> For 2.x, the only methods that need to be overridden explicitly are those
> where this bound method caching does the wrong thing (__exit__ and __enter__
> needed to be on that list, which is what first brought this class to my
> attention). For 3.0, it was also necessary to add:
>
> def __iter__(self):
> return iter(self.file)
>
> It wasn't too bad in this case since file doesn't implement many tp_*
> slots, but the 3.0 version of classes that delegate a lot of operations to a
> specific member variable will be a lot more verbose in any cases where the
> underlying type being delegated to implements some of the number or
> container protocols.
>
>
>
> >
> > > to new-style, all that would then be needed is to invoke the new method
> on
> > > the class object after defining the class (a class decorator or
> > > metaclass could be provided somewhere to make this a bit tidier).
> > >
> >
> > Hm. So you are thinking of a proxy for a class?!?!
> >
>
> Sort of - I'm thinking mainly of classes like _TemporaryFileWrapper that
> delegate most operations to a specific member variable, and expect that
> member variable to always be of a specific type.
>
>
>
> > Note that if you set a class attribute corresponding to a special
> > method (e.g. C.__add__ = ...) the corresponding C slot is
> > automatically updated, so you should be able to write a class
> > decorator or mixin or helper function to do this in pure Python,
> > unless I completely misunderstand what you're after.
> >
>
> Yeah, doing it in typeobject was mostly an easy way of getting at the
> complete list of special methods with tp_* slots without having to maintain
> two copies of that list.
>
>
>
> >
> > > This seems a lot cleaner than expecting everyone that implements a
> proxy
> > > object to maintain there own list of all of the relevant special
> > > methods, and locates the implementation support in an area of the code
> > > that already has plenty of infrastructure dedicated to keeping Python
> > > visible attributes in sync with the C visible tp_* slots.
> > >
> >
> > How many proxy implementations does the world need? Maybe we should
> > add one to the stdlib?
> >
>
> I don't know enough about the different ways people proxy or otherwise
> delegate special methods to know if it is feasible to provide a
> one-size-fits-most implementation in the standard library.
>
> That said, maybe it would be enough if a type instance could be queried for
> the list of special method names it implements that the interpreter can
> access without going through __getattribute__?
>
> Then the slots of a class delegating to a specific type could be
> initialised appropriately by doing something like:
>
> for name in delegate_type.special_methods():
> if not hasattr(cls, name):
> def delegation(*args, **kwds):
> self, *args = args # +1 on arbitrary tuple unpacking ;)
> getattr(self.delegate, name)(*args, **kwds)
> setattr(cls, name, delegation)
>
> The approach I suggested in my original email would instead look more like
> this:
>
> class Foo: ...
>
> Foo.delegate_special_methods('delegate', delegate_type)
>
> where delegate_special_methods is basically just a C level implementation
> of the loop described above (except that the 'delegation' callable could be
> a lot more efficient than the given Python function).
>
> Another option would be to provide an explicit list in the documentation of
> the slot names for the tp_* methods which the interpreter may access without
> going through __getattr__ and __getattribute__.
>
> The discussion in the bug report that got me thinking about this topic
> commented on the fact that quite a few magic methods were added during the
> 2.x development cycle - I think the key point I missed at the time is the
> fact that most of those *didn't* have corresponding tp_* slots, so
> __getattr__ and __getattribute__ (particularly the latter) can intercept
> them just fine.
>
> That said, the documentation approach would probably be too limiting on
> alternate interpreters though - why should other implementations be
> restricted from providing optimised access to special methods just because
> we haven't done so certain cases in CPython?
>
> If we don't make any changes at all, the delegation loop shown above can
> actually already be written as follows:
>
> for name in dir(delegate_type):
> if (name.startswith('__')
> and name.endswith('__')
> and not hasattr(cls, name)):
> def delegation(*args, **kwds):
> self, *args = args # +1 on arbitrary tuple unpacking ;)
> getattr(self.delegate, name)(*args, **kwds)
> setattr(cls, name, delegation)
>
>
>
> >
> > > Thoughts? Altenative ideas? Howls of protest?
> > >
> >
> > No, so far just a bit of confusion. :-)
> >
>
> Hopefully the above makes my concerms a bit clearer.
>
> I'm actually hoping to hear from some more people that would benefit from
> having better support for this kind of delegation - my interest in the
> matter is fairly academic (based solely on the tempfile bugs arising from
> the initial conversion to Py3k), so my personal inclination is actually to
> put a stronger note in the documentation about the fact that the lookup of
> special methods may bypass __getattribute__ entirely and leave it at that.
>
> Cheers,
> Nick.
>
>
>
> --
> Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
> ---------------------------------------------------------------
> http://www.boredomandlaziness.org
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
_______________________________________________
Python-3000 mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
