Re: Subclassing Python's dict
On Thu, Aug 6, 2009 at 1:19 PM, alex23 wuwe...@gmail.com wrote: Xavier Ho wrote: You should subclass collections.UserDict, and not the default dict class. Refer to the collections module. Xavier, why do you think that is the correct approach? I'll be honest first and say that I do not completely understand how dict is implemented in the underlying C structure. But as Bruno had already mentioned, dict has a slightly different behaviour then we'd expect. For example, the __getitem__() function isn't actually used by the interpreter (which, you know, *can* be a problem.) http://www.python.org/download/releases/2.2.3/descrintro/#subclassing (I don't know if 2.6.2 changed anything since 2.2.3, but there are some references you can look at, and sample code.) To answer your question, it's really not the correct approach, but I think (meaning: untested) UserDict doesn't have the same implementation, which might free up some restrictions Sergey encountered. The docs say The need for this class has been largely supplanted by the ability to subclass directly from dict (a feature that became available starting with Python version 2.2). I didn't realise they took UserDict out in later versions (2.6, for example), and put it back in Python 3.0. Does anyone know why? So, I do not actually know. If you mean that Sergey should subclass _in this instance_ could you please explain why? (Sorry if you already have, I never saw your original post...) It was really more or less an educated guess. He didn't tell us what he is trying to achieve, so _in this instance_ I can't give more advice than saying well, if this doesn't work, here are you other options that I know of/just found. Try these and let us know if you got it to work. *winks* - Sometimes I do answer a little too quickly, I apologise for that. - Xav -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
On Aug 5, 7:39 am, Bruno Desthuilliers bruno. 42.desthuilli...@websiteburo.invalid wrote: Sergey Simonenko a écrit : Hi, I subclass builtin 'dict' in my application and experience some problems with it. The whole issue is that I should redefine 'setdefault' and 'update' methods after redefining '__setitem__' or/and '__delitem__', otherwise 'update' and 'setdefault' ignore redefined '__setitem__' and use builtin dict's one so dict looks kinda like a black box. Another guy have reported me that he experiences similar problems with subclassing builtin 'list'. I indeed notice this behaviour here (Python 2.6.2). I'm afraid it has to do with some optimization tricks (dict being the very fundamental data structure in Python, it has to be higly optimized). You can ultimately blame it on optimization in this case, but in general there isn't any way (in Python) to require that __setitem__ be the common point for all modifications. In fact this is an example of a general condition of OOP. A high- level method may or may not call a low-level method to do its dirty work, and there is no way to know this simply from the class's interface. You need to have knowledge of the implementation to know whether you need to override high-level or low-level methods. In general you can't safely subclass based on the interface alone; you have to know what you're subclassing. (As an aside: this intuitively is the reason why I never cared much for some of the arguments against inheritance. Many arguments against inheritance go like this: Inheritance is bad because X can happen if you're not careful. Doesn't mean anything to me because you have be careful anyway.) Because of the care required when subclassing, I tend to use inheritance only when the intended base class is A. also under my control, or B. specifically designed to be subclassed. Examples of B in the standard Python library are Queue and Thread. list and dict are allowed to be subclassed but aren't specifically designed for it, so I tend not to subclass those. (It is occasionally indispensible, though.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
Are you referring to Python 3.0? Python 2.6 does not have collections.UserDict In Python2.6, it is in its own module. from UserDict import UserDict Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
Xavier Ho wrote: You should subclass collections.UserDict, and not the default dict class. Refer to the collections module. Xavier, why do you think that is the correct approach? The docs say The need for this class has been largely supplanted by the ability to subclass directly from dict (a feature that became available starting with Python version 2.2). UserDict can be a good choice because the pure python source makes it clear exactly what needs to be overridden (you can see which methods are implemented in terms of lower level methods and which ones access the underlying dict directly. Another choice is to use DictMixin or MutableMapping and fill-in just the required abstract methods. This approach is simple and flexible. It allows you to wrap a mapping interface around many different classes (a dbm for example). The disadvantage is that it can be slow. Subclassing a dict is typically done when the new class has to be substitutable for real dicts (perhaps an API enforces a check for instance(x, dict) or somesuch). As the OP found out, the dict methods all access the underlying structure directly, so you will need to override *all* methods that need to have a new behavior. The remaining methods are inherited and run at C speed, so performance may dictate this approach. So, there you have three ways to do it. In Py3.1, we used the latter approach for collections.Counter() -- that gives a high speed on the inherited methods. For collections.OrderedDict, a hybrid approach was used (subclassing from both dict and MutableMapping). Most of the work is done by MutableMapping and the dict is inherited so that the OrderedDict objects would be substitutable anywhere regular dicts are expected. And IIRC, there are still some cases of UserDict being used in the python source (situations where subclassing from dict wouldn't work as well). Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
En Thu, 06 Aug 2009 05:26:22 -0300, Xavier Ho cont...@xavierho.com escribió: On Thu, Aug 6, 2009 at 1:19 PM, alex23 wuwe...@gmail.com wrote: Xavier Ho wrote: You should subclass collections.UserDict, and not the default dict class. Xavier, why do you think that is the correct approach? I'll be honest first and say that I do not completely understand how dict is implemented in the underlying C structure. But as Bruno had already mentioned, dict has a slightly different behaviour then we'd expect. For example, the __getitem__() function isn't actually used by the interpreter (which, you know, *can* be a problem.) Thinks have evolved... Before Python 2.2, builtin types were not subclassable. You could not inherit from dict. In order to write another mapping class, you had to write the complete interface - or inherit from UserDict, that was a concrete class implementing the mapping protocol. Later, DictMixin was added (in 2.3) and it made easier to write other mapping classes: one had to write the most basic methods (__getitem__ / __setitem__ / __delitem__ / keys) and the DictMixin provided the remaining functionality (e.g. values() is built from keys() plus __getitem__). Later releases allowed an even more modular approach, and until 2.5 DictMixin was the recommended approach. Then came 3.0/2.6 and PEP3119 defining a rich hierarchy of abstract base classes; a normal dictionary implements the MutableMapping ABC and this is the preferred approach now (the MutableMapping implementation is very similar to the original DictMixin, but builds on the other base classes like Sized, Iterable...) I didn't realise they took UserDict out in later versions (2.6, for example), and put it back in Python 3.0. Does anyone know why? UserDict still exists on both releases (collections.UserDict on 3.x), but it's not the preferred approach to implement a new mapping class anymore. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Subclassing Python's dict
Hi, I subclass builtin 'dict' in my application and experience some problems with it. The whole issue is that I should redefine 'setdefault' and 'update' methods after redefining '__setitem__' or/and '__delitem__', otherwise 'update' and 'setdefault' ignore redefined '__setitem__' and use builtin dict's one so dict looks kinda like a black box. Another guy have reported me that he experiences similar problems with subclassing builtin 'list'. Kind regards, Sergey. -- Написано в почтовом клиенте браузера Opera: http://www.opera.com/mail/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
On Thu, Aug 6, 2009 at 11:51 AM, Sergey Simonenko gfo...@lavabit.comwrote: I subclass builtin 'dict' in my application and experience some problems with it. You should subclass collections.UserDict, and not the default dict class. Refer to the collections module. Also, the ABC MutableMapping might be of your interest. Another guy have reported me that he experiences similar problems with subclassing builtin 'list'. Similarly, UserList is what you should subclass. HTH, Ching-Yun Xavier Ho, Technical Artist Contact Information Mobile: (+61) 04 3335 4748 Skype ID: SpaXe85 Email: cont...@xavierho.com Website: http://xavierho.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
Sergey Simonenko a écrit : Hi, I subclass builtin 'dict' in my application and experience some problems with it. The whole issue is that I should redefine 'setdefault' and 'update' methods after redefining '__setitem__' or/and '__delitem__', otherwise 'update' and 'setdefault' ignore redefined '__setitem__' and use builtin dict's one so dict looks kinda like a black box. Another guy have reported me that he experiences similar problems with subclassing builtin 'list'. I indeed notice this behaviour here (Python 2.6.2). I'm afraid it has to do with some optimization tricks (dict being the very fundamental data structure in Python, it has to be higly optimized). -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
Xavier Ho wrote: On Thu, Aug 6, 2009 at 11:51 AM, Sergey Simonenko gfo...@lavabit.comwrote: I subclass builtin 'dict' in my application and experience some problems with it. You should subclass collections.UserDict, and not the default dict class. Refer to the collections module. Are you referring to Python 3.0? Python 2.6 does not have collections.UserDict j -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
On Thu, Aug 6, 2009 at 5:23 AM, Joshua Kugler jos...@joshuakugler.comwrote: Are you referring to Python 3.0? Python 2.6 does not have collections.UserDict j Yes, I was sometimes it's hard to keep track what's not in 2.6 and in 3.1 for me, sorry. And thanks. The ABC MutableMapping is still valid, though. I hope he gets it sorted out. With a morning yawn, Xavier -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing Python's dict
Xavier Ho wrote: You should subclass collections.UserDict, and not the default dict class. Refer to the collections module. Xavier, why do you think that is the correct approach? The docs say The need for this class has been largely supplanted by the ability to subclass directly from dict (a feature that became available starting with Python version 2.2). If you mean that Sergey should subclass _in this instance_ could you please explain why? (Sorry if you already have, I never saw your original post...) -- http://mail.python.org/mailman/listinfo/python-list
