Re: Is this a good use of __metaclass__?
Hi! Thanks for taking the time to answer. I will definitely have a look at writing dispatchers. The problem you have with your metaclass version, is the infamous metaclass conflict. I think I solved the problem of conflicting metaclasses in this case and I posted it as a reply to Bruno Desthuilliers in this thread. Do you also think that's a bad use of __metaclass__? It's not that I'm hellbent on using metaclasses - I'm just curious how people think they should be used. Cheers! /Joel -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a good use of __metaclass__?
[EMAIL PROTECTED] wrote: Hi! Thank you for a quick and informative response! I'd go for 'manually decorating' anyway. Metaclasses can be really handy for framework-like stuff, but for the use case you describe, I think the explicit decorator option is much more, well, explicit - and also more flexible - than metaclass black magic. Yes, point taken. This may also help you distinguish 'published' API from implementation (which is what CherryPy do) Hmm... I'm not sure I understand how manually decorating would help me do that? With SimpleXMLRPCServer.register_instance(obj) all public methods of obj are published for XMLRPC, decorated or not. So it's effectively useless (disclaimer : I've never used SimpleXMLRPCServer). (snip) You would then have a 'server' class that just provides common services and dispatch to specialized objects. Neat. It won't play nice with dir() or SimpleXMLRPCServer's introspection functions though (system.listMethods(), system.methodHelp()). That may be a showstopper, or do you know of any fixes? Nope - well, at least not without digging into SimpleXMLRPCServer's internals, but if it (- the solution I suggested) makes things more complicated, it's a bad idea anyway. -- bruno desthuilliers python -c print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p in '[EMAIL PROTECTED]'.split('@')]) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a good use of __metaclass__?
[EMAIL PROTECTED] wrote: I played around with my old code before I saw your post, and I believe I've found a solution that's a bit neater than what I had before. I thought I could just as well post it if you're interested and have the time. This one uses multiple inheritance, but it's legal and there's only one metaclass. (snip code) Seems quite clean. -- bruno desthuilliers python -c print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p in '[EMAIL PROTECTED]'.split('@')]) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a good use of __metaclass__?
Joel Hedlund wrote: It's not that I'm hellbent on using metaclasses - I'm just curious how people think they should be used. There are very few good use cases for metaclasses. *You should use a metaclass only when you want the additional metaclass-induced behavior to be inherited in the children classes.* Here is a made up example (it assume you know the cmd.Cmd class in the standard library). Suppose you want to use a metaclasses to add aliases to the do_* methods of a subclass of cmd.Cmd. You can do it in this way: import cmd class make_oneletter_aliases_for_commands(type): Typically used in Cmd classes. def __init__(cls, name, bases, dic): for name,func in dic.iteritems(): if name.startswith(do_): firstletter = name[3] setattr(cls, do_ + firstletter, func) class Cmd(cmd.Cmd, object): # make it a new-style class, so that super works __metaclass__ = make_oneletter_aliases_for_commands def preloop(self): Done once at the beginning. cmd.Cmd.preloop(self) self.do_greetings(World!) def do_greetings(self, arg): Display a greeting message. print Hello, %s! % arg def do_help(self, arg): Print this help message. super(Cmd, self).do_help(arg) def do_quit(self,arg): Exit the command-loop. return 'quit' # anything != None will do if __name__ == '__main__': Cmd().cmdloop() Here is how you use it: $ python cmd_with_aliases.py Hello, World!! (Cmd) h Documented commands (type help topic): g greetings h help q quit (Cmd) g World Hello, World! (Cmd) q The metaclass has generated the methods 'do_h', 'do_g' and 'do_q' as aliases for 'do_help', 'do_greetings', 'do_quit' respectively. You could have done the same without a metaclass, with a function modifying the class. However every time you subclass Cmd (or a subclass of it), you would have to invoke the function again to generate the aliases corresponding to the new methods in the subclass. The metaclass performs this step automagically, simplifying the life of your users. If you don't care about giving magic abilities to the subclasses of subclasses you don't need a metaclass. For real use of metaclasses, see the Zope framework code. Michele Simionato -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a good use of __metaclass__?
Answering to the title of your post, no, this is not a good use of metaclasses. Your problem seems a textbook example of multiple dispatch, so I suggest you to look at PEAK with has an implementation of multimethods/generic functions. Notice that Guido seems to be intentioned to add support for generic functions in future versions of Python, so that solution would likely have the blessing of the BDFL ;) The problem you have with your metaclass version, is the infamous metaclass conflict. It can be solved by hand or automatically (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/204197) but it is best to avoid it altogether. Just use PEAK or an hand made dispatcher, like for instance this one: class SimpleDispatcher(object): # this is on purpose not object oriented A dispatcher is a callable object that looks in a namespace for callable objects and calls them with the signature ``dispatcher(callablename, dispatchtag, *args, **kw)`` The namespace can be a module, a class, a dictionary, or anything that responds to ``getattr`` or (alternatively) to ``__getitem__``. Here is an example of usage: call = SimpleDispatcher(globals()) def manager_showpage(): ...return 'Manager' def member_showpage(): ... return 'Member' def anonymous_showpage(): ... return 'Anonymous' call('showpage', 'anonymous') 'Anonymous' call('showpage', 'manager') 'Manager' call('showpage', 'member') 'Member' def __init__(self, ns): self._ns = ns def __call__(self, funcname, classname, *args, **kw): try: func = getattr(self._ns, '%s_%s' % (classname, funcname)) except AttributeError: func = self._ns['%s_%s' % (classname, funcname)] return func(*args, **kw) if __name__ == __main__: import doctest; doctest.testmod() BTW, the usual advice holds here: if you can find an workable solution not involving metaclasses and decorators, don't use them. Michele Simionato -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a good use of __metaclass__?
Hi! Thank you for a quick and informative response! I'd go for 'manually decorating' anyway. Metaclasses can be really handy for framework-like stuff, but for the use case you describe, I think the explicit decorator option is much more, well, explicit - and also more flexible - than metaclass black magic. Yes, point taken. This may also help you distinguish 'published' API from implementation (which is what CherryPy do) Hmm... I'm not sure I understand how manually decorating would help me do that? With SimpleXMLRPCServer.register_instance(obj) all public methods of obj are published for XMLRPC, decorated or not. Off course, I could write a @do_not_publish decorator, but that seems backwards to me... I'm not familiar with how CherryPy works either, I'm sorry to say. And finally, this may let you organize your code more freely - you can mix methods needing different decorators in a same class. One can still do that, even if one would use a metaclass to set the bare necessities decorators. All you have to do is add the extra ones manually. I just meant to let the metaclass do the really, really important ones for me (the validator for each API class). The ones that I can't, won't, mustn't forget to add lest the Script Kiddies of the Internet come brutalizing my data. :-) You would then have a 'server' class that just provides common services and dispatch to specialized objects. Neat. It won't play nice with dir() or SimpleXMLRPCServer's introspection functions though (system.listMethods(), system.methodHelp()). That may be a showstopper, or do you know of any fixes? My 2 cents... Thanks! Those were what I was hoping for, after all. Thanks for your help! /Joel -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a good use of __metaclass__?
I played around with my old code before I saw your post, and I believe I've found a solution that's a bit neater than what I had before. I thought I could just as well post it if you're interested and have the time. This one uses multiple inheritance, but it's legal and there's only one metaclass. When executed, this prints: Validating admin. Scary stuff done. Cheers! /Joel Here ya go! -- from decorator import decorator # Metaclass for decorating public methods: class DecoratePublicMethods(type): Equip public methods of a class with a specified decorator. Class data members: decorator_attribute = '_public_method_decorator': str If this attribute of the class exists and evaluates to True, then it is used to decorate all public methods of the class. no_decoration_attribute = '_do_not_decorate': str If this attribute of the class exists it should contain a list of names of public methods that should not be decorated. decorator_attribute = '_public_method_decorator' no_decoration_attribute = '_do_not_decorate' def __new__(cls, classname, bases, classdict): decorator = classdict.get(cls.decorator_attribute, None) if not decorator: return type.__new__(cls,classname,bases,classdict) do_not_decorate = classdict.get(cls.no_decoration_attribute, []) for attr,item in classdict.items(): if not callable(item): continue if attr in do_not_decorate or attr.startswith('_'): continue classdict[attr] = decorator(item) return type.__new__(cls, classname, bases, classdict) # Some decorators: @decorator def validate_job(func, self, id, response, *pargs, **kwargs): Bogus authentiction routine print Validating job. return func(self, id, response, *pargs, **kwargs) @decorator def validate_user(func, self, id, response, *pargs, **kwargs): Bogus authentiction routine print Validating user. return func(self, id, response, *pargs, **kwargs) @decorator def validate_admin(func, self, id, response, *pargs, **kwargs): Bogus authentiction routine print Validating admin. return func(self, id, response, *pargs, **kwargs) # My API: class BaseAPI(object): __metaclass__ = DecoratePublicMethods class JobValidatedAPI(BaseAPI): _public_method_decorator = validate_job def do_routine_stuff(self, clientid, response, foo): print Routine stuff done. class UserValidatedAPI(BaseAPI): _public_method_decorator = validate_user def do_mundane_stuff(self, clientid, response, moo): print Mundane stuff done. class AdminValidatedAPI(BaseAPI): _public_method_decorator = validate_admin def do_scary_stuff(self, clientid, response, moose): print Scary stuff done. ## FIXED: Multiple inheritance now legal. class FullAPI(JobValidatedAPI, UserValidatedAPI, AdminValidatedAPI): _public_method_decorator = None # Now this works: b = FullAPI() b.do_scary_stuff('bofh', 2, 3) -- -- http://mail.python.org/mailman/listinfo/python-list
Is this a good use of __metaclass__?
Hi! I need some input on my use of metaclasses since I'm not sure I'm using them in a pythonic and graceful manner. I'm very grateful for any tips, pointers and RTFMs I can get from you guys. Below, you'll find some background info and an executable code example. In the code example I have two ways of doing the same thing. The problem is that the Neat version doesn't work, and the Ugly version that works gives me the creeps. The Neat version raises a TypeError when I try the multiple inheritance (marked with comment in the code): Traceback (most recent call last): File /bioinfo/yohell/pybox/gridjs/gridjs-2.0/test.py, line 132, in ? class FullAPI(JobAPI, UserAPI, AdminAPI): File /bioinfo/yohell/pybox/gridjs/gridjs-2.0/test.py, line 43, in __new__ return type.__new__(cls,classname,bases,classdict) TypeError: Error when calling the metaclass bases metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases In the Ugly version, I'm changing the metaclass in the global scope between class definitions, and that gives me bad vibes. What should I do? Is there a way to fix my Neat solution? Is my Ugly solution in fact not so horrid as I think it is? Or should I rethink the whole idea? Or maybe stick with decorating manually (or in BaseAPI.__init__)? Sincere thanks for your time. Cheers! /Joel Hedlund Background ## (feel free to skip this if you are in a hurry) I'm writing an XMLRPC server that serves three types of clients (jobs, users and admins). To do this I'm subclassing SimpleXMLRPCServer for all the connection work, and I was planning on putting the entire XMLRPC API as public methods of a class, and expose it to clients using .register_instance(). Each session is initiated by a handshake where a challenge is presented to the client, each method call must then be authenticated using certificates and incremental digest response. Each client type must be authenticated differently, and each type of client will also use a discrete set of methods. At first, I was planning to use method decorators to do the validation, and have a different decorator for each type of client validation, like so: class FullAPI: @valid_job_required def do_routine_stuff(self, clientid, response, ...): pass @valid_user_required def do_mundane_stuff(self, clientid, response, ...): pass @valid_admin_required def do_scary_stuff(self, clientid, response, ...): pass ... There will be a lot of methods for each client type, so this class would become monstrous. Therefore I started wondering if it weren't a better idea to put the different client APIs in different classes and decorate them separately using metaclasses, and finally bring the APIs together using multiple inheritance. This is what I had in mind: class BaseAPI(object): pass class JobApi(BaseAPI): pass class UserApi(BaseAPI): pass class AdminApi(BaseAPI): pass class FullApi(JobAPI, UserAPI, AdminAPI): pass Now, I'm having trouble implementing the metaclass bit in a nice and pythonic way. Code example test.py === # Base metaclass for decorating public methods: from decorator import decorator @decorator def no_change(func, *pargs, **kwargs): return func(*pargs, **kwargs) class DecoratePublicMethods(type): Equip all public methods with a given decorator. Class data members: decorator = no_change: decorator The decorator that you wish to apply to public methods of the class instances. The default does not change program behavior. do_not_decorate = []: iterable str Names of public methods that should not be decorated. multiple_decoration = False: bool If set to False, methods will not be decorated if they already have been decorated by a prior metaclass. decoration_tag = '__public_method_decorated__': str Decorated public methods will be equipped with an attribute with this name and a value of True. decorator = no_change do_not_decorate = [] multiple_decoration = True decoration_tag = '__public_method_decorated__' def __new__(cls,classname,bases,classdict): for attr,item in classdict.items(): if not callable(item): continue if attr in cls.do_not_decorate or attr.startswith('_'): continue if (not cls.multiple_decoration and hasattr(classdict[attr], cls.decoration_tag)): continue classdict[attr] = cls.decorator(item) setattr(classdict[attr], cls.decoration_tag, True) return type.__new__(cls,classname,bases,classdict) ## Authentication stuff: class AuthenticationError(Exception): pass import random
Re: Is this a good use of __metaclass__?
Joel Hedlund [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Below, you'll find some background info and an executable code example. In the code example I have two ways of doing the same thing. The problem is that the Neat version doesn't work, and the Ugly version that works gives me the creeps. To me, your Ugly version is easier to read and understand than the so-called Neat version. You change an attribute of a class to change its behavior without changing its identity. I can imagine ways to hide this, but I would leave it simple and explicit. Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a good use of __metaclass__?
Joel Hedlund a écrit : Hi! I need some input on my use of metaclasses since I'm not sure I'm using them in a pythonic and graceful manner. I'm very grateful for any tips, pointers and RTFMs I can get from you guys. Below, you'll find some background info and an executable code example. In the code example I have two ways of doing the same thing. The problem is that the Neat version doesn't work, and the Ugly version that works gives me the creeps. The Neat version raises a TypeError when I try the multiple inheritance (marked with comment in the code): Traceback (most recent call last): File /bioinfo/yohell/pybox/gridjs/gridjs-2.0/test.py, line 132, in ? class FullAPI(JobAPI, UserAPI, AdminAPI): File /bioinfo/yohell/pybox/gridjs/gridjs-2.0/test.py, line 43, in __new__ return type.__new__(cls,classname,bases,classdict) TypeError: Error when calling the metaclass bases metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/204197 In the Ugly version, I'm changing the metaclass in the global scope between class definitions, Yuck. and that gives me bad vibes. What should I do? Is there a way to fix my Neat solution? Is my Ugly solution in fact not so horrid as I think it is? Or should I rethink the whole idea? Or maybe stick with decorating manually (or in BaseAPI.__init__)? I'd go for 'manually decorating' anyway. Metaclasses can be really handy for framework-like stuff, but for the use case you describe, I think the explicit decorator option is much more, well, explicit - and also more flexible - than metaclass black magic. This may also help you distinguish 'published' API from implementation (which is what CherryPy do). And finally, this may let you organize your code more freely - you can mix methods needing different decorators in a same class. Also, and FWIW, you may want to favor composition-delegation (which is a piece of cake in Python, see below...) over inheritance. This is more flexible. You would then have a 'server' class that just provides common services and dispatch to specialized objects. class Dispatcher(object): def __init__(self, delegates): self._delegates = delegates def __getattr__(self, name): for delegate in self._delegates: try: return getattr(delegate, name): except AttributeError: pass else: err = object '%s' has no attribute '%s' \ % (self.__class__.__name__, name) raise AttributeError(err) # note that you may want to reorganize your code differently # if you go for explicit decorators d = Dispatcher(JobApi, UserApi, AdminApi) My 2 cents... -- http://mail.python.org/mailman/listinfo/python-list