New issue 2436: Support pybind11 in conjunction with PyPy's cpyext: Crash in PyDict_Next https://bitbucket.org/pypy/pypy/issues/2436/support-pybind11-in-conjunction-with-pypys
Wenzel Jakob: Hi, following the fix in Issue #2434, I was able to get basic types & methods to export properly in [pybind11](https://github.com/pybind/pybind11). Also, the entire test suite compiles now (it segfaults when actually running it, but still..). Yay! :) Looking into the test suite crashes, it turns out that the culprit is a call to ``PyDict_Next()`` to traverse the contents of a newly created type. Pybind11 uses this function to iterate over the dictionary of a type when binding C++ enumerations in Python (to do some basic postprocessing of the enumeration entries). The CPython API calls boil down to this: ```cpp PyObject *dict = ((PyTypeObject *) ... /* Pointer to newly created type */)->tp_dict; PyObject *key, *value; ssize_t pos = 0; while (PyDict_Next(dict, &pos, &key, &value)) { /* Do something -- Never gets here because PyDict_Next crashes */ } ``` This works fine in Python 2.7 and 3.x but crashes in PyPy. What's weird is that ``dict`` is a perfectly good dictionary even in PyPy. I can for instance turn it into a string and print it before the ``PyDict_Next()`` call, getting ```python {'__init__': <unbound method pybind11_tests.EMode.__init__>, '__new__': <builtin_function_or_method object at 0x0000000104754250>, '__dict__': <getset_descriptor object at 0x00000001019724d8>, '__weakref__': <getset_descriptor object at 0x000000010191b068>, '__doc__': None, '__module__': 'pybind11_tests', '__repr__': <unbound method pybind11_tests.EMode.__repr__>, '__int__': <unbound method pybind11_tests.EMode.__int__>, '__eq__': <unbound method pybind11_tests.EMode.__eq__>, '__ne__': <unbound method pybind11_tests.EMode.__ne__>, '__hash__': <unbound method pybind11_tests.EMode.__hash__>, '__getstate__': <unbound method pybind11_tests.EMode.__getstate__>, '__setstate__': <unbound method pybind11_tests.EMode.__setstate__>, 'EFirstMode': EMode.EFirstMode, 'ESecondMode': EMode.ESecondMode} ``` but ``PyDict_Next()`` still crashes. LLDB reports the following backtrace: ``` * frame #0: 0x0000000000000000 frame #1: 0x000000010062c0a4 libpypy-c.dylib`pypy_g_PyDict_Next + 244 frame #2: 0x00000001005f479b libpypy-c.dylib`pypy_g_wrapper_second_level__star_4_4 + 347 frame #3: 0x0000000105d8b0e0 pybind11_tests.pypy-41.so`pybind11::enum_<MyEnum>::export_values(this=0x00007fff5fbfe580) + 80 at pybind11.h:1260 ``` which is bizarre. If I read that correctly, PyPy seems to have done a jump to NULL. How to reproduce on your end: ``` $ git clone https://github.com/wjakob/pybind11 $ cd pybind11 $ cmake -DPYTHON_EXECUTABLE:FILEPATH=<path-to-pypy> $ make $ pypy -m pip install pytest $ pypy -m pytest $ ``` Thank you, Wenzel _______________________________________________ pypy-issue mailing list [email protected] https://mail.python.org/mailman/listinfo/pypy-issue
