[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > I think that's all with this issue. Thanks Demur and Serhiy for your great work! -- ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: I think that's all with this issue. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: Thanks for the report, warning fixed in the issue #28105. -- ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Objects/methodobject.c: In function ‘_PyCFunction_FastCallKeywords’: Objects/methodobject.c:281:24: warning: ‘nkwargs’ may be used uninitialized in this function [-Wmaybe-uninitialized] assert((nargs == 0 && nkwargs == 0) || stack != NULL);

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > Why don't we change the magic number before b1 (if your bytecode change is > already committed)? The magic number changed many times between Python 3.5 and the future Python 3.6 beta 1. Maybe we skipped increasing this number for one bytecode change, but

[issue27213] Rework CALL_FUNCTION* opcodes

Yury Selivanov added the comment: > After releasing beta 1 this would require changing bytecode magic number. Why don't we change the magic number before b1 (if your bytecode change is already committed)? -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Roundup Robot added the comment: New changeset e372c0ad32ce by Victor Stinner in branch 'default': Revert change f860b7a775c5 https://hg.python.org/cpython/rev/e372c0ad32ce New changeset 2558bc4a4ebf by Victor Stinner in branch 'default': Document kwnames in _PyObject_FastCallKeywords() and

[issue27213] Rework CALL_FUNCTION* opcodes

Roundup Robot added the comment: New changeset f860b7a775c5 by Victor Stinner in branch 'default': ssue #27213: Reintroduce checks in _PyStack_AsDict() https://hg.python.org/cpython/rev/f860b7a775c5 -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Roundup Robot added the comment: New changeset f217419d08f0 by Victor Stinner in branch 'default': Issue #27213: Fix reference leaks https://hg.python.org/cpython/rev/f217419d08f0 -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: The change 51b635e81958 introduced memory leaks. Example: $ ./python -m test -R 3:3 test_extcall (...) test_extcall leaked [102, 102, 102] references, sum=306 test_extcall leaked [41, 41, 41] memory blocks, sum=123 I'm analyzing the change to try to identify

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: I have committed the patch before beta 1 because it change bytecode generating and interpreting. The bytecode generated before the patch is compatible with the patched interpreter, but the bytecode generated by patched compiler can be not compatible with

[issue27213] Rework CALL_FUNCTION* opcodes

Robert Collins added the comment: See http://bugs.python.org/issue28086 - this introduced a regression in the test suite. -- nosy: +rbcollins ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Roundup Robot added the comment: New changeset 51b635e81958 by Serhiy Storchaka in branch 'default': Issue #27213: Fixed different issues with reworked CALL_FUNCTION* opcodes. https://hg.python.org/cpython/rev/51b635e81958 -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Changes by Serhiy Storchaka : Removed file: http://bugs.python.org/file44557/IMG_20160830_174452.jpg ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

Changes by Влад Ключкей : -- hgrepos: +354 Added file: http://bugs.python.org/file44557/IMG_20160830_174452.jpg ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Please attach a reproducer Berker. I still can't reproduce crashes. Maybe it depends on platform or installed third-party module or Python command line options. -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Berker Peksag added the comment: Thanks, Serhiy. test_recursionlimit_recovery in test_sys passed, but now I'm getting the following failure: FAIL: test_recursionlimit_fatalerror (test.test_sys.SysModuleTest) I modified test_recursionlimit_fatalerror to use subTest() and it looks like the

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: > python: Objects/abstract.c:2385: _PyStack_AsDict: Assertion > `PyDict_GetItem(kwdict, key) != ((void *)0)' failed. Aborted (core dumped) > generate-posix-vars failed > make: *** [pybuilddir.txt] Error 1 Ah, it's my fault. I didn't test the latest patch with

[issue27213] Rework CALL_FUNCTION* opcodes

Berker Peksag added the comment: > [...] please try a "make distclean" and make sure that all .pyc files are > removed, and then recompile Python. I forgot to mention that in my earlier comment. I usually run "make distclean" before "./configure --with-pydebug; make -s -j" so unfortunately

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-09-11 08:55 Berker: "I'm getting segfaults with a77756e480c2 (thanks to 'hg bisect') on Ubuntu 12.04 (...) test_recursionlimit_recovery (...) test_main_recursion_error (...)" We

[issue27213] Rework CALL_FUNCTION* opcodes

Berker Peksag added the comment: I'm getting segfaults with a77756e480c2 (thanks to 'hg bisect') on Ubuntu 12.04. I've checked buildbots, but they seem happy. In test_sys: test_recursionlimit_recovery (test.test_sys.SysModuleTest) ... Fatal Python error: Segmentation fault In test_runpy:

[issue27213] Rework CALL_FUNCTION* opcodes

Changes by Serhiy Storchaka : Added file: http://bugs.python.org/file44546/callfunc-11.patch ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > Here is rebased patch. callfunc-10.patch looks like an even better enhancement compared to Python 3.5, nice work. Would you be ok to wait after the beta1 release? The change doesn't seem to change the bytecode too much, nor the C API, whereas I'm trying to

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Here is rebased patch. -- Added file: http://bugs.python.org/file44510/callfunc-10.patch ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: I just pushed my patch for the issue #27830 (add _PyObject_FastCallKeywords()). I'm trying to even push the first parts of METH_FASTCALL (issue #27810). Could you please try to rebase callfunc-9.patch on top of the development branch? The code is moving s

[issue27213] Rework CALL_FUNCTION* opcodes

Roundup Robot added the comment: New changeset 854b08acca97 by Victor Stinner in branch 'default': Issue #27213: document changes in Misc/NEWS https://hg.python.org/cpython/rev/854b08acca97 -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: I'm really sorry Serhiy that I didn't give you more time to finish the review and push yourself the change :-( Feel free to enhance the code, I don't expect major changes. For test_traceback, I opened the issue #28050: if you already know how to fix it,

[issue27213] Rework CALL_FUNCTION* opcodes

Roundup Robot added the comment: New changeset a77756e480c2 by Victor Stinner in branch 'default': Rework CALL_FUNCTION* opcodes https://hg.python.org/cpython/rev/a77756e480c2 -- nosy: +python-dev ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > I just fixed test_extcall and test_traceback, and going to get rid of the > duplication in _PyEval_EvalCodeWithName. I need maybe a hour or two. What do you mean? I already fixed test_extcall and code duplication? "I fixed test_extcall and removed

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: I just fixed test_extcall and test_traceback, and going to get rid of the duplication in _PyEval_EvalCodeWithName. I need maybe a hour or two. -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > Please wait. I just have sat down at my computer. Can you please give me a status and/or an ETA? This change is blocking me for 2 more changes. -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Please wait. I just have sat down at my computer. -- ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > Please wait. I just have sat down at my computer. Oh, you are here :-) Go ahead! -- ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > I'm going to push the patch today. I'm going to push the patch in ~10 min since I don't see it pushed yet. > But changes to test_extcall look as a regression to me. I agree that it looks like a regression, but it might be tricky to retrieve the function

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: I'm going to push the patch today. But changes to test_extcall look as a regression to me. -- ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

Changes by Serhiy Storchaka : -- assignee: -> serhiy.storchaka ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > Known issues: > * test_traceback and test_extcall fail > * _PyEval_EvalCodeWithName2() should be merged into > _PyEval_EvalCodeWithName() to factorize the code I fixed test_extcall and removed _PyEval_EvalCodeWithName2(): callfunc-8.patch. I don't know how

[issue27213] Rework CALL_FUNCTION* opcodes

Yury Selivanov added the comment: Serhiy, Victor and I want this to be merged tomorrow morning PST to land in 3.6 before the freeze. Would you be able to merge this yourself? If not, we'll do it ourselves in the morning. -- nosy: +yselivanov ___

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: Rebased change. -- Added file: http://bugs.python.org/file44485/callfunc-7.patch ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: I uploaded callfunc-6.patch which is supposed to be a rebased version of callfunc5.patch. I made tiny coding style changes. Python/ceval.c changed a lot since 1 month 1/2, so I'm not 100% sure that the rebase is correct. Known issues: * test_traceback and

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: I'm working on a new calling convention: "fast call". I already pushed changes implementing first functions: _PyObject_FastCallDict(PyObject *func, PyObject **stack, Py_ssize_t nargs, PyObject *kwargs) kwargs is a Python dict, but it can be NULL.

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: > Pybench is now only ~200ms slower instead of 1200ms slower. But the whole > point of this patch is that CALL_FUNCTION_EX shouldn't be optimized for, so > I'd much prefer real benchmarking results FYI I released a first version of the "new" Python benchmark

[issue27213] Rework CALL_FUNCTION* opcodes

STINNER Victor added the comment: See also the issue #27809 "_PyObject_FastCall(): add support for keyword arguments" where we discuss if we can completely avoid the creation of a temporary dictionary to pass keyword arguments. -- ___ Python

[issue27213] Rework CALL_FUNCTION* opcodes

Demur Rumed added the comment: Since the most common use of CALL_FUNCTION_EX is.. def f(*x,*kw): other_func(*x, **kw) I've added some code to BUILD_MAP_UNPACK_WITH_CALL & BUILD_TUPLE_UNPACK to not allocate a new object if called with oparg of 1 & TOP() is correct type -- Added

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: The ABI of BUILD_MAP_UNPACK_WITH_CALL is changed. oparg=514 meant merging 2 dicts, but with the patch it will mean merging 514 dicts. The INCREF/DECREF calls on function objects surrounding calls are not needed, because the refcount was increased when the

[issue27213] Rework CALL_FUNCTION* opcodes

Demur Rumed added the comment: callfunc3 addresses most feedback. Doesn't address _PyEval_EvalCodeWithName2 code bloat, & I disagree with mentioning BUILD_MAP_UNPACK_WITH_CALL change in magic number update as the ABI of BUILD_MAP_UNPACK_WITH_CALL is unchanged. ie if we were to implement

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Added comments on Rietveld. -- ___ Python tracker ___ ___

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Issue27358 is too complex, more complex than this issue. But I think the simple fix the regression in case of the single **kw is checking wherever the sum dict is empty. -if (with_call) { +if (with_call && PyDict_Size(sum)) {

[issue27213] Rework CALL_FUNCTION* opcodes

Demur Rumed added the comment: callfunc2 fixes test_dis, addresses code review, currently implements a copy of _PyEval_EvalCodeWithName as _PyEval_EvalCodeWithName2 which changes a few lines to work with new keyword stack layout so that we can use fast_function with kwargs CALL_FUNCTION_EX

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Thank you Demur. I left few comments on Rietveld (this is only the quick glance). Good news that this change allows to simplify BUILD_MAP_UNPACK_WITH_CALL. But using 15th bit means using EXTENDED_ARG. I think it would be simpler to just push an empty tuple

[issue27213] Rework CALL_FUNCTION* opcodes

Changes by Serhiy Storchaka : -- stage: -> patch review ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

Mark Shannon added the comment: I seemed to have been added to the nosy list. I guess that means that my opinions are wanted ;) I have wanted to clean up the code around making calls for quite a while. I just haven't had time. I agree that adding a simpler opcode for making calls with

[issue27213] Rework CALL_FUNCTION* opcodes

Demur Rumed added the comment: Attaching first iteration. Very drafty. Still need to fix test_dis; will run test suite this evening. Perf on pybench went from 16.5s to 17.5s. It was 18.3s prior to reintroducing use of fast_function. Code still needs clean up besides investigation into how to

[issue27213] Rework CALL_FUNCTION* opcodes

Demur Rumed added the comment: I've been working on this, may have the ceval portion mostly worked out but can't test until I finish the compile portion. Haven't had time this week, will have time to focus this weekend -- ___ Python tracker

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: Do you take this for your Demur? -- ___ Python tracker ___ ___

[issue27213] Rework CALL_FUNCTION* opcodes

Serhiy Storchaka added the comment: I was going to write a patch myself, but since I'm sure in your skills, I yield it to you. -- ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

Demur Rumed added the comment: I'd like to take on creating a patch for this proposal once #27140 lands -- ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

Changes by Serhiy Storchaka : -- nosy: +Demur Rumed, Mark.Shannon ___ Python tracker ___

[issue27213] Rework CALL_FUNCTION* opcodes

New submission from Serhiy Storchaka: Currently there are 4 opcodes (CALL_FUNCTION, CALL_FUNCTION_VAR, CALL_FUNCTION_KW, CALL_FUNCTION_VAR_KW) for calling a function depending of presenting the var-positional and var-keyword arguments: func(arg1, ..., argN, name1=kwarg1, ...,

[issue27213] Rework CALL_FUNCTION* opcodes

Changes by Serhiy Storchaka : -- dependencies: +Opcode for creating dict with constant keys ___ Python tracker ___