Well, I discussed this issue hundreds of times with Victor Stinner. I believe that this is what we have to go even if there is a very little minor performance issue, it will be not a big hurdle. we can see the benchmark from https://speed.python.org/ and CPython become faster and faster. Converting macros to functions will overcome the issue which has been pointed out as known as implementation detail leak.
Regards, Dong-hee 2021년 10월 20일 (수) 오전 9:59, Victor Stinner <vstin...@python.org>님이 작성: > Hi, > > Erlend and me wrote a PEP to move away from macros in the Python C > API. We are now waiting for feedback :-) Read the PEP online: > https://www.python.org/dev/peps/pep-0670/ > > There is a copy of the PEP below for inline replies. > > Victor > > --- > > PEP: 670 > Title: Convert macros to functions in the Python C API > Author: Erlend Egeberg Aasland <erlend.aasl...@protonmail.com>, > Victor Stinner <vstin...@python.org> > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 19-Oct-2021 > Python-Version: 3.11 > > > Abstract > ======== > > Convert macros to static inline functions or regular functions. > > Remove the return value of macros having a return value, whereas they > should not, to aid detecting bugs in C extensions when the C API is > misused. > > Some function arguments are still cast to ``PyObject*`` to prevent > emitting new compiler warnings. > > > Rationale > ========= > > The use of macros may have unintended adverse effects that are hard to > avoid, even for experienced C developers. Some issues have been known > for years, while others have been discovered recently in Python. > Working around macro pitfalls makes the macro coder harder to read and > to maintain. > > Converting macros to functions has multiple advantages: > > * By design, functions don't have macro pitfalls. > * Arguments type and return type are well defined. > * Debuggers and profilers can retrieve the name of inlined functions. > * Debuggers can put breakpoints on inlined functions. > * Variables have a well defined scope. > * Code is usually easier to read and to maintain than similar macro > code. Functions don't need the following workarounds for macro > pitfalls: > > * Add parentheses around arguments. > * Use line continuation characters if the function is written on > multiple lines. > * Add commas to execute multiple expressions. > * Use ``do { ... } while (0)`` to write multiple statements. > > Converting macros and static inline functions to regular functions makes > these regular functions accessible to projects which use Python but > cannot use macros and static inline functions. > > > Macro Pitfalls > ============== > > The `GCC documentation > <https://gcc.gnu.org/onlinedocs/cpp/Macro-Pitfalls.html>`_ lists several > common macro pitfalls: > > - Misnesting > - Operator precedence problems > - Swallowing the semicolon > - Duplication of side effects > - Self-referential macros > - Argument prescan > - Newlines in arguments > > > Performance and inlining > ======================== > > Static inline functions is a feature added to the C99 standard. Modern C > compilers have efficient heuristics to decide if a function should be > inlined or not. > > When a C compiler decides to not inline, there is likely a good reason. > For example, inlining would reuse a register which require to > save/restore the register value on the stack and so increase the stack > memory usage or be less efficient. > > > Debug build > ----------- > > When Python is built in debug mode, most compiler optimizations are > disabled. For example, Visual Studio disables inlining. Benchmarks must > not be run on a Python debug build, only on release build: using LTO and > PGO is recommended for reliable benchmarks. PGO helps the compiler to > decide if function should be inlined or not. > > > Force inlining > -------------- > > The ``Py_ALWAYS_INLINE`` macro can be used to force inlining. This macro > uses ``__attribute__((always_inline))`` with GCC and Clang, and > ``__forceinline`` with MSC. > > So far, previous attempts to use ``Py_ALWAYS_INLINE`` didn't show any > benefit and were abandoned. See for example: `bpo-45094 > <https://bugs.python.org/issue45094>`_: "Consider using > ``__forceinline`` and ``__attribute__((always_inline))`` on static > inline functions (``Py_INCREF``, ``Py_TYPE``) for debug build". > > When the ``Py_INCREF()`` macro was converted to a static inline > functions in 2018 (`commit > < > https://github.com/python/cpython/commit/2aaf0c12041bcaadd7f2cc5a54450eefd7a6ff12 > >`__), > it was decided not to force inlining. The machine code was analyzed with > multiple C compilers and compiler options: ``Py_INCREF()`` was always > inlined without having to force inlining. The only case where it was not > inlined was the debug build. See discussion in the `bpo-35059 > <https://bugs.python.org/issue35059>`_: "Convert ``Py_INCREF()`` and > ``PyObject_INIT()`` to inlined functions". > > > Disable inlining > ---------------- > > On the other side, the ``Py_NO_INLINE`` macro can be used to disable > inlining. It is useful to reduce the stack memory usage. It is > especially useful on a LTO+PGO build which is more aggressive to inline > code: see `bpo-33720 <https://bugs.python.org/issue33720>`_. The > ``Py_NO_INLINE`` macro uses ``__attribute__ ((noinline))`` with GCC and > Clang, and ``__declspec(noinline)`` with MSC. > > > Specification > ============= > > Convert macros to static inline functions > ----------------------------------------- > > Most macros should be converted to static inline functions to prevent > `macro pitfalls`_. > > The following macros should not be converted: > > * Empty macros. Example: ``#define Py_HAVE_CONDVAR``. > * Macros only defining a number, even if a constant with a well defined > type can better. Example: ``#define METH_VARARGS 0x0001``. > * Compatibility layer for different C compilers, C language extensions, > or recent C features. > Example: ``#define Py_ALWAYS_INLINE __attribute__((always_inline))``. > > > Convert static inline functions to regular functions > ---------------------------------------------------- > > The performance impact of converting static inline functions to regular > functions should be measured with benchmarks. If there is a significant > slowdown, there should be a good reason to do the conversion. One reason > can be hiding implementation details. > > Using static inline functions in the internal C API is fine: the > internal C API exposes implemenation details by design and should not be > used outside Python. > > Cast to PyObject* > ----------------- > > When a macro is converted to a function and the macro casts its > arguments to ``PyObject*``, the new function comes with a new macro > which cast arguments to ``PyObject*`` to prevent emitting new compiler > warnings. So the converted functions still accept pointers to structures > inheriting from ``PyObject`` (ex: ``PyTupleObject``). > > For example, the ``Py_TYPE(obj)`` macro casts its ``obj`` argument to > ``PyObject*``:: > > #define _PyObject_CAST_CONST(op) ((const PyObject*)(op)) > > static inline PyTypeObject* _Py_TYPE(const PyObject *ob) { > return ob->ob_type; > } > #define Py_TYPE(ob) _Py_TYPE(_PyObject_CAST_CONST(ob)) > > The undocumented private ``_Py_TYPE()`` function must not be called > directly. Only the documented public ``Py_TYPE()`` macro must be used. > > Later, the cast can be removed on a case by case basis, but that is out > of scope for this PEP. > > Remove the return value > ----------------------- > > When a macro is implemented as an expression, it has an implicit return > value. In some cases, the macro must not have a return value and can be > misused in third party C extensions. See `bpo-30459 > <https://bugs.python.org/issue30459>`_ for the example of > ``PyList_SET_ITEM()`` and ``PyCell_SET()`` macros. It is not easy to > notice this issue while reviewing macro code. > > These macros are converted to functions using the ``void`` return type > to remove their return value. Removing the return value aids detecting > bugs in C extensions when the C API is misused. > > > Backwards Compatibility > ======================= > > Removing the return value of macros is an incompatible API change made > on purpose: see the `Remove the return value`_ section. > > > Rejected Ideas > ============== > > Keep macros, but fix some macro issues > -------------------------------------- > > Converting macros to functions is not needed to `remove the return > value`_: casting a macro return value to ``void`` also fix the issue. > For example, the ``PyList_SET_ITEM()`` macro was already fixed like > that. > > Macros are always "inlined" with any C compiler. > > The duplication of side effects can be worked around in the caller of > the macro. > > People using macros should be considered "consenting adults". People who > feel unsafe with macros should simply not use them. > > Examples of hard to read macros > =============================== > > _Py_NewReference() > ------------------ > > Example showing the usage of an ``#ifdef`` inside a macro. > > Python 3.7 macro (simplified code):: > > #ifdef COUNT_ALLOCS > # define _Py_INC_TPALLOCS(OP) inc_count(Py_TYPE(OP)) > # define _Py_COUNT_ALLOCS_COMMA , > #else > # define _Py_INC_TPALLOCS(OP) > # define _Py_COUNT_ALLOCS_COMMA > #endif /* COUNT_ALLOCS */ > > #define _Py_NewReference(op) ( \ > _Py_INC_TPALLOCS(op) _Py_COUNT_ALLOCS_COMMA \ > Py_REFCNT(op) = 1) > > Python 3.8 function (simplified code):: > > static inline void _Py_NewReference(PyObject *op) > { > _Py_INC_TPALLOCS(op); > Py_REFCNT(op) = 1; > } > > PyObject_INIT() > --------------- > > Example showing the usage of commas in a macro. > > Python 3.7 macro:: > > #define PyObject_INIT(op, typeobj) \ > ( Py_TYPE(op) = (typeobj), _Py_NewReference((PyObject *)(op)), > (op) ) > > Python 3.8 function (simplified code):: > > static inline PyObject* > _PyObject_INIT(PyObject *op, PyTypeObject *typeobj) > { > Py_TYPE(op) = typeobj; > _Py_NewReference(op); > return op; > } > > #define PyObject_INIT(op, typeobj) \ > _PyObject_INIT(_PyObject_CAST(op), (typeobj)) > > The function doesn't need the line continuation character. It has an > explicit ``"return op;"`` rather than a surprising ``", (op)"`` at the > end of the macro. It uses one short statement per line, rather than a > single long line. Inside the function, the *op* argument has a well > defined type: ``PyObject*``. > > > References > ========== > > * `bpo-45490 <https://bugs.python.org/issue45490>`_: > [meta][C API] Avoid C macro pitfalls and usage of static inline > functions (October 2021). > * `What to do with unsafe macros > <https://discuss.python.org/t/what-to-do-with-unsafe-macros/7771>`_ > (March 2021). > * `bpo-43502 <https://bugs.python.org/issue43502>`_: > [C-API] Convert obvious unsafe macros to static inline functions > (March 2021). > > > Copyright > ========= > > This document is placed in the public domain or under the > CC0-1.0-Universal license, whichever is more permissive. > -- > Night gathers, and now my watch begins. It shall not end until my death. > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/2GN646CGWGTO6ZHHU7JTA5XWDF4ULM77/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Software Development Engineer at Line corp. Tel: +82 10-3353-9127 Email: donghee.n...@gmail.com | donghee...@python.org | donghee...@linecorp.com Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JMT7TH37GMUKJNHVCZ7RXXPTPAXKQ2PG/ Code of Conduct: http://python.org/psf/codeofconduct/
