Here is my first try with metaclasses in py3. It does not handle
__prepare__ I don't understand what function should I call to create
cell... and what is that func
2010/11/4 Stefan Behnel <[email protected]>:
> Vitja Makarov, 03.11.2010 23:03:
>> 2010/11/4 Stefan Behnel:
>>> Vitja Makarov, 03.11.2010 22:13:
>>>> Now it seems to me that class declaration is more like function call
>>>> with special case for metaclass keyword and positional arguments:
>>>>
>>>> class Base(type):
>>>> def __new__(cls, name, bases, attrs, **kwargs):
>>>> print(cls, name, bases, attrs, kwargs)
>>>>
>>>> class Foo(1, 2, 3, metaclass=Base, foo='bar'):
>>>> def hello(self):
>>>> pass
>>>
>>> Yes, I think that's exactly how this should work. Want to give it another
>>> try?
>>
>> Yes, I want to finish this part ;)
>
> Great! :)
>
> BTW, thanks for doing all this. Even your first patch was pretty good for a
> "first patch".
>
>
>>> It seems that the number of positional arguments is fixed, but you can pass
>>> any number of keyword arguments, out of which only "metaclass" is special
>>> cased and removed (from a copy of the dict, BTW). There is also an
>>> additional protocol if the metaclass has a "__prepare__" class method. See
>>> "__build_class__".
>>
>> Yes, all positional arguments are turned into bases. So we can make it this
>> way:
>>
>> Pyx_CreateClass(PyObject *bases, PyObject *dict, PyObject *name,
>> PyObject *modname, PyObject *kwargs);
>>
>> And then
>>
>> args = (metaclass, bases, name, dict)
>> PyEval_CallObjectWithKeywords(metaclass, args, kwargs)
>
> Except for the first 'metaclass' in args (args has length 3). Again, see
> the __build_class__ implementation.
>
>
>>> I think the Python 2 protocol in Cython should work as in Py3. Metaclasses
>>> in Py2 won't generally support kwargs, but if a user explicitly provides
>>> them, it's best to assume that that's intentional. In the worst case, there
>>> will be a runtime TypeError.
>>
>> Do you mean that new syntax should be used for Py2?
>
> No, just what happens behind the syntax. A __metaclass__ field can easily
> be mapped to a metaclass keyword argument (or the equivalent representation
> on a PyClassDefNode in Cython) and from that point on, it's the same thing
> in both cases.
>
>
>>> So, kwargs should pass through, except for __metaclass__. If both the
>>> __metaclass__ class field and the metaclass kwarg are passed, I'd lean
>>> towards ignoring the field and prefer the kwarg, just like Py3 does. It can
>>> be argued that this is worth a warning, though.
>>>
>> Py3 doesn't support __metaclass__ attribute, so if metaclass is set that
>> seems to be mostly Py3 so __metaclass__ should be ignored.
>
> If I'm not mistaken, the __metaclass__ field can always be handled by the
> compiler during type analysis (or maybe in a transform, see the classes in
> ParseTreeTransforms.py, for example). You can't dynamically add an
> attribute to a class at definition time without letting the parser see its
> name. So you just have to check the class body and you'll know if the field
> is there or not.
>
> Cython also has a Python 3 syntax mode (look out for "language_level") in
> which we can disable this field lookup, so that you get semantic
> equivalence with Python 3 even before the code generation phase. So we
> don't need any runtime support for the __metaclass__ field, not even in Py2.
>
>
>>> I also think that the parser should extract the metaclass keyword, not the
>>> runtime code. So, if provided, it should be passed into __Pyx_CreateClass()
>>> as an argument and not in the kwargs.
>>
>> I think parser can't extract metaclass keyword, try this code:
>>
>> class Base(type):
>> def __new__(cls, name, bases, attrs, **kwargs):
>> print(cls, name, bases, attrs, kwargs)
>>
>> myargs={'metaclass': Base}
>> class Foo(1, 2, 3, **myargs):
>> def hello(self):
>> pass
>
> Well, it can if it's passed explicitly, though.
>
> We already need to support an explicit metaclass to handle the
> __metaclass__ field. In most cases, the keyword argument will be spelled
> out explicitly, so we can reduce the runtime overhead in that case. But
> that sounds like an optimisation, not a required feature.
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> [email protected]
> http://codespeak.net/mailman/listinfo/cython-dev
>
diff -r 4b5abd81bb75 Cython/Compiler/ExprNodes.py
--- a/Cython/Compiler/ExprNodes.py Wed Nov 03 21:44:55 2010 +0100
+++ b/Cython/Compiler/ExprNodes.py Thu Nov 04 14:13:02 2010 +0300
@@ -4475,14 +4475,23 @@
# dict ExprNode Class dict (not owned by this node)
# doc ExprNode or None Doc string
# module_name EncodedString Name of defining module
-
- subexprs = ['bases', 'doc']
+ # keyword_args ExprNode or None Py3 Dict of keyword arguments, passed to __new__
+ # starstar_arg ExprNode or None Py3 Dict of extra keyword args, same here
+
+ subexprs = ['bases', 'keyword_args', 'starstar_arg', 'doc']
def analyse_types(self, env):
self.bases.analyse_types(env)
if self.doc:
self.doc.analyse_types(env)
self.doc = self.doc.coerce_to_pyobject(env)
+ if self.keyword_args:
+ self.keyword_args.analyse_types(env)
+ if self.starstar_arg:
+ self.starstar_arg.analyse_types(env)
+ if self.starstar_arg:
+ self.starstar_arg = \
+ self.starstar_arg.coerce_to_pyobject(env)
self.type = py_object_type
self.is_temp = 1
env.use_utility_code(create_class_utility_code);
@@ -4496,6 +4505,19 @@
def generate_result_code(self, code):
cname = code.intern_identifier(self.name)
+ if self.keyword_args and self.starstar_arg:
+ code.put_error_if_neg(self.pos,
+ "PyDict_Update(%s, %s)" % (
+ self.keyword_args.py_result(),
+ self.starstar_arg.py_result()))
+ keyword_code = self.keyword_args.py_result()
+ elif self.keyword_args:
+ keyword_code = self.keyword_args.py_result()
+ elif self.starstar_arg:
+ keyword_code = self.starstar_arg.py_result()
+ else:
+ keyword_code = 'NULL'
+
if self.doc:
code.put_error_if_neg(self.pos,
'PyDict_SetItemString(%s, "__doc__", %s)' % (
@@ -4503,12 +4525,13 @@
self.doc.py_result()))
py_mod_name = self.get_py_mod_name(code)
code.putln(
- '%s = __Pyx_CreateClass(%s, %s, %s, %s); %s' % (
+ '%s = __Pyx_CreateClass(%s, %s, %s, %s, %s); %s' % (
self.result(),
self.bases.py_result(),
self.dict.py_result(),
cname,
py_mod_name,
+ keyword_code,
code.error_goto_if_null(self.result(), self.pos)))
code.put_gotref(self.py_result())
@@ -7154,44 +7177,73 @@
create_class_utility_code = UtilityCode(
proto = """
-static PyObject *__Pyx_CreateClass(PyObject *bases, PyObject *dict, PyObject *name, PyObject *modname); /*proto*/
+static PyObject *__Pyx_CreateClass(PyObject *bases, PyObject *dict, PyObject *name,
+ PyObject *modname, PyObject *kwargs); /*proto*/
""",
impl = """
static PyObject *__Pyx_CreateClass(
- PyObject *bases, PyObject *methods, PyObject *name, PyObject *modname)
+ PyObject *bases, PyObject *dict, PyObject *name, PyObject *modname, PyObject *kwargs)
{
- PyObject *result = 0;
-#if PY_MAJOR_VERSION < 3
- PyObject *metaclass = 0, *base;
-#endif
-
- if (PyDict_SetItemString(methods, "__module__", modname) < 0)
+ PyObject *result = NULL;
+ PyObject *metaclass = NULL;
+
+ if (PyDict_SetItemString(dict, "__module__", modname) < 0)
goto bad;
- #if PY_MAJOR_VERSION < 3
- metaclass = PyDict_GetItemString(methods, "__metaclass__");
-
- if (metaclass != NULL)
- Py_INCREF(metaclass);
- else if (PyTuple_Check(bases) && PyTuple_GET_SIZE(bases) > 0) {
- base = PyTuple_GET_ITEM(bases, 0);
- metaclass = PyObject_GetAttrString(base, "__class__");
- if (metaclass == NULL) {
- PyErr_Clear();
- metaclass = (PyObject *)base->ob_type;
+
+ /* Python3 metaclasses */
+ if (kwargs) {
+ metaclass = PyDict_GetItemString(kwargs, "metaclass");
+ if (metaclass) {
Py_INCREF(metaclass);
+ /* it is safe to modify kwargs here */
+ if (PyDict_DelItemString(kwargs, "metaclass") < 0) {
+ Py_DECREF(metaclass);
+ goto bad;
+ }
}
}
- else {
- metaclass = (PyObject *) &PyClass_Type;
+
+ /* Python2 metaclasses */
+ if (metaclass == NULL) {
+ metaclass = PyDict_GetItemString(dict, "__metaclass__");
+ if (metaclass)
+ Py_INCREF(metaclass);
+ }
+
+ if (metaclass == NULL) {
+#if PY_MAJOR_VERSION < 3
+ if (PyTuple_Check(bases) && PyTuple_GET_SIZE(bases) > 0) {
+ PyObject *base = PyTuple_GET_ITEM(bases, 0);
+ metaclass = PyObject_GetAttrString(base, "__class__");
+ if (metaclass == NULL) {
+ PyErr_Clear();
+ metaclass = (PyObject *)base->ob_type;
+ }
+ } else {
+ metaclass = (PyObject *) &PyClass_Type;
+ }
+#else
+ if (PyTuple_Check(bases) && PyTuple_GET_SIZE(bases) > 0) {
+ PyObject *base = PyTuple_GET_ITEM(bases, 0);
+ metaclass = (PyObject *)base->ob_type;
+ } else {
+ metaclass = (PyObject *) &PyType_Type;
+ }
+
+#endif
Py_INCREF(metaclass);
}
- result = PyObject_CallFunctionObjArgs(metaclass, name, bases, methods, NULL);
+ if (kwargs && PyDict_Size(kwargs) > 0) {
+ PyObject *margs;
+ margs = PyTuple_Pack(3, name, bases, dict, NULL);
+ result = PyEval_CallObjectWithKeywords(metaclass, margs, kwargs);
+ Py_DECREF(margs);
+ } else {
+ result = PyObject_CallFunctionObjArgs(metaclass, name, bases, dict, NULL);
+ }
+
Py_DECREF(metaclass);
- #else
- /* it seems that python3+ handle __metaclass__ itself */
- result = PyObject_CallFunctionObjArgs((PyObject *)&PyType_Type, name, bases, methods, NULL);
- #endif
bad:
return result;
}
diff -r 4b5abd81bb75 Cython/Compiler/Nodes.py
--- a/Cython/Compiler/Nodes.py Wed Nov 03 21:44:55 2010 +0100
+++ b/Cython/Compiler/Nodes.py Thu Nov 04 14:13:02 2010 +0300
@@ -2935,7 +2935,8 @@
child_attrs = ["body", "dict", "classobj", "target"]
decorators = None
- def __init__(self, pos, name, bases, doc, body, decorators = None):
+ def __init__(self, pos, name, bases, doc, body, decorators = None,
+ keyword_args = None, starstar_arg = None):
StatNode.__init__(self, pos)
self.name = name
self.doc = doc
@@ -2950,7 +2951,8 @@
else:
doc_node = None
self.classobj = ExprNodes.ClassNode(pos, name = name,
- bases = bases, dict = self.dict, doc = doc_node)
+ bases = bases, dict = self.dict, doc = doc_node,
+ keyword_args = keyword_args, starstar_arg = starstar_arg)
self.target = ExprNodes.NameNode(pos, name = name)
def as_cclass(self):
diff -r 4b5abd81bb75 Cython/Compiler/Parsing.py
--- a/Cython/Compiler/Parsing.py Wed Nov 03 21:44:55 2010 +0100
+++ b/Cython/Compiler/Parsing.py Thu Nov 04 14:13:02 2010 +0300
@@ -381,7 +381,7 @@
# arglist: argument (',' argument)* [',']
# argument: [test '='] test # Really [keyword '='] test
-def p_call(s, function):
+def p_call_parse(s):
# s.sy == '('
pos = s.position()
s.next()
@@ -428,29 +428,43 @@
if s.sy == ',':
s.next()
s.expect(')')
+ return positional_args, keyword_args, star_arg, starstar_arg
+
+def p_call_prepare_full(pos, positional_args, keyword_args, star_arg):
+ arg_tuple = None
+ keyword_dict = None
+ if positional_args or not star_arg:
+ arg_tuple = ExprNodes.TupleNode(pos,
+ args = positional_args)
+ if star_arg:
+ star_arg_tuple = ExprNodes.AsTupleNode(pos, arg = star_arg)
+ if arg_tuple:
+ arg_tuple = ExprNodes.binop_node(pos,
+ operator = '+', operand1 = arg_tuple,
+ operand2 = star_arg_tuple)
+ else:
+ arg_tuple = star_arg_tuple
+ if keyword_args:
+ keyword_args = [ExprNodes.DictItemNode(pos=key.pos, key=key, value=value)
+ for key, value in keyword_args]
+ keyword_dict = ExprNodes.DictNode(pos,
+ key_value_pairs = keyword_args)
+ return arg_tuple, keyword_dict
+
+def p_call(s, function):
+ # s.sy == '('
+ pos = s.position()
+
+ positional_args, keyword_args, star_arg, starstar_arg = \
+ p_call_parse(s)
+
if not (keyword_args or star_arg or starstar_arg):
return ExprNodes.SimpleCallNode(pos,
function = function,
args = positional_args)
else:
- arg_tuple = None
- keyword_dict = None
- if positional_args or not star_arg:
- arg_tuple = ExprNodes.TupleNode(pos,
- args = positional_args)
- if star_arg:
- star_arg_tuple = ExprNodes.AsTupleNode(pos, arg = star_arg)
- if arg_tuple:
- arg_tuple = ExprNodes.binop_node(pos,
- operator = '+', operand1 = arg_tuple,
- operand2 = star_arg_tuple)
- else:
- arg_tuple = star_arg_tuple
- if keyword_args:
- keyword_args = [ExprNodes.DictItemNode(pos=key.pos, key=key, value=value)
- for key, value in keyword_args]
- keyword_dict = ExprNodes.DictNode(pos,
- key_value_pairs = keyword_args)
+ arg_tuple, keyword_dict = p_call_prepare_full(pos,
+ positional_args, keyword_args, star_arg)
return ExprNodes.GeneralCallNode(pos,
function = function,
positional_args = arg_tuple,
@@ -2607,16 +2621,23 @@
s.next()
class_name = EncodedString( p_ident(s) )
class_name.encoding = s.source_encoding
+ arg_tuple = None
+ keyword_dict = None
+ starstar_arg = None
if s.sy == '(':
- s.next()
- base_list = p_simple_expr_list(s)
- s.expect(')')
- else:
- base_list = []
+ positional_args, keyword_args, star_arg, starstar_arg = \
+ p_call_parse(s)
+ arg_tuple, keyword_dict = p_call_prepare_full(pos,
+ positional_args, keyword_args, star_arg)
+ if arg_tuple is None:
+ # XXX: empty arg_tuple
+ arg_tuple = ExprNodes.TupleNode(pos, args = [])
doc, body = p_suite(s, Ctx(level = 'class'), with_doc = 1)
return Nodes.PyClassDefNode(pos,
name = class_name,
- bases = ExprNodes.TupleNode(pos, args = base_list),
+ bases = arg_tuple,
+ keyword_args = keyword_dict,
+ starstar_arg = starstar_arg,
doc = doc, body = body, decorators = decorators)
def p_c_class_definition(s, pos, ctx):
diff -r 4b5abd81bb75 tests/run/metaclass.pyx
--- a/tests/run/metaclass.pyx Wed Nov 03 21:44:55 2010 +0100
+++ b/tests/run/metaclass.pyx Thu Nov 04 14:13:02 2010 +0300
@@ -11,3 +11,23 @@
True
"""
__metaclass__ = Base
+
+class Py3Base(type):
+ def __new__(cls, name, bases, attrs, foo=None):
+ attrs['foo'] = foo
+ return type.__new__(cls, name, bases, attrs)
+
+ def __init__(self, cls, attrs, obj, foo=None):
+ super(Py3Base, self).__init__(cls, attrs, obj)
+# NOT YET SUPPORTED
+# @staticmethod
+# def __prepare__(name, bases, **kwargs):
+# return {'bar': 666}
+
+class Py3Foo(object, metaclass=Py3Base, foo=123):
+ """
+ >>> obj = Py3Foo()
+ >>> obj.foo
+ 123
+ """
+ zzz = 111
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
