Re: [Python-Dev] Deleting with setting C API functions
On 3 December 2015 at 02:35, Guido van Rossum wrote: > On Wed, Dec 2, 2015 at 7:32 AM, Emanuel Barry wrote: >> >> Nick Coghlan made a pretty elaborated blog post about that here: >> http://opensource.com/life/14/9/why-python-4-wont-be-python-3 > > I wholeheartedly agree with what Nick writes there -- but I can't resist > noting that the title is backwards -- the whole point is that Python 4 > *will* be like Python 3, i.e. it will *not* differ (in a > backward-incompatible way) from Python 3. What Nick probably meant is "Why > the *transition to* Python 4 won't be like the transition to Python 3." And > that is exactly right. Yeah, the full title was actually "Why Python 4.0 won't be like Python 3.0" which I think better conveys the implied "transition to" aspect, but the zeros got left out in the opensource.com URL. The RHEL dev blog version uses the full title in the URL as well: https://developerblog.redhat.com/2014/09/17/why-python-4-0-wont-be-like-python-3-0/ I also wrote the article before you mentioned you might be amenable to doing a 3.10 instead of rolling over to 4.0 (although I suspect there are even more systems that assume XY is sufficient to identify a Python feature release than assumed XYZ was sufficient to identify maintenance releases prior to 2.7.10) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 3/12/2015 5:41 a.m., Random832 wrote: Why bother with the dot? Why not rename 3.5 to Python 5, and then go to Python 6, etc, and then your "4.0" would be 10. Then we could call it Python X! Everything is better with an X in the name. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 2015-12-02, Andrew Barnert wrote: > Python could just go from 3.9 to 4.0, as a regular dot release, just > to dispel the idea of an inevitable backward-incompatible "Python 4". > (That should be around 2 years after the expiration of 2.7 support, > py2/py3 naming, etc., right?) Why bother with the dot? Why not rename 3.5 to Python 5, and then go to Python 6, etc, and then your "4.0" would be 10. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On Wed, Dec 2, 2015 at 7:32 AM, Emanuel Barry wrote: > Nick Coghlan made a pretty elaborated blog post about that here: > http://opensource.com/life/14/9/why-python-4-wont-be-python-3 > I wholeheartedly agree with what Nick writes there -- but I can't resist noting that the title is backwards -- the whole point is that Python 4 *will* be like Python 3, i.e. it will *not* differ (in a backward-incompatible way) from Python 3. What Nick probably meant is "Why the *transition to* Python 4 won't be like the transition to Python 3." And that is exactly right. We've learned our lesson (though we're in much better shape than Perl :-). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On Dec 2, 2015, at 07:01, Random832 wrote: > > On 2015-12-02, Victor Stinner wrote: >>> Are there plans for a Python 4? >> >> No. Don't. Don't schedule any "removal" or *any* kind of "break >> backward compatibility" anymore, or you will definetly kill the Python >> community. > > I feel like I should note that I agree with your position here, I was > just asking the question to articulate the issue that "put it off to the > indefinite future" isn't a real plan for anything. Python could just go from 3.9 to 4.0, as a regular dot release, just to dispel the idea of an inevitable backward-incompatible "Python 4". (That should be around 2 years after the expiration of 2.7 support, py2/py3 naming, etc., right?) Or, of course, Python could avoid the number 4, go to 3.17 and then decide that the next release is big enough to be worthy of 5.0. Or go from 3.9 to 2022, or XP, or Python Enterprise Python 1. :) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
> From: victor.stin...@gmail.com > Date: Wed, 2 Dec 2015 15:46:37 +0100 > To: random...@fastmail.com > Subject: Re: [Python-Dev] Deleting with setting C API functions > CC: python-dev@python.org > > 2015-12-02 15:40 GMT+01:00 Random832 : > > Are there plans for a Python 4? > > No. Don't. Don't schedule any "removal" or *any* kind of "break > backward compatibility" anymore, or you will definetly kill the Python > community. Nick Coghlan made a pretty elaborated blog post about that here: http://opensource.com/life/14/9/why-python-4-wont-be-python-3 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 2015-12-02, Victor Stinner wrote: >> Are there plans for a Python 4? > > No. Don't. Don't schedule any "removal" or *any* kind of "break > backward compatibility" anymore, or you will definetly kill the Python > community. I feel like I should note that I agree with your position here, I was just asking the question to articulate the issue that "put it off to the indefinite future" isn't a real plan for anything. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
2015-12-02 15:40 GMT+01:00 Random832 : > Are there plans for a Python 4? No. Don't. Don't schedule any "removal" or *any* kind of "break backward compatibility" anymore, or you will definetly kill the Python community. It will probably take 10 years or more to convert *all* Python 2 code around the world to Python 3. I don't want to have to redo the same thing again. Never ever again. To be clear: removing functions is fine, but if and only if you have a smooth transition plan. Sorry, it's unclear to me what is a "smooth transition plan". IMHO the deprecation warnings which are current quiet by default is not a good idea. Everybody ignore them, and then complain when the function is really removed. Maybe I should write an informal PEP to explain my idea. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 2015-12-02, M.-A. Lemburg wrote: > A first step would be to replace the macro with a proper function > to avoid false positive warnings, even when using the correct API. > > Then we could add a warning to the PyObject_SetAttr() function and > hope that not too many projects use the stable ABI as basis to > have C extensions work across several releases. How about using a versioned ABI? Make a new function that doesn't allow NULL, called something like PyObject_SetAttr2, and instead of declaring the old one in headers, use a #define to the new name. > Overall, I'm not sure whether it's worth the trouble. Documenting > the feature and adding a deprecation notice to just the documentation > would likely be better. We could then remove the functionality > in Python 4. Are there plans for a Python 4? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 2 December 2015 at 22:41, M.-A. Lemburg wrote: > On 02.12.2015 13:29, Serhiy Storchaka wrote: >> On 02.12.15 12:06, Victor Stinner wrote: >>> 2015-12-02 9:42 GMT+01:00 Serhiy Storchaka : You have enough time to update your projects, and you can update them uniformly for all versions. And may be you will found few weird bugs related to misuse of Set* API. >>> >>> Did you check popular projects using C extensions to check if they >>> call Set*() functions to delete attributes/items? >> >> I have checked following projects. >> >> regex, simplejson, Pillow, PyQt4, LibreOffice, PyGTK, PyICU, pyOpenSSL, >> libxml2, Boost, psutil, >> mercurial don't use PyObject_SetAttr at all. >> >> NumPy, pgobject don't use PyObject_SetAttr for deleting. >> >> PyYAML and lxml use PyObject_SetAttr only in code generated by Cython and >> never use it for deleting. > > While I don't think deleting attributes is a very common thing > to do in any Python code base (unless you need to break circular > references or explicitly want to free resources), the > fact that PyObject_DelAttr() itself is implemented as macro > using the NULL attribute value clearly creates an API incompatibility > when removing this functionality or generating warnings, since > all code using the correct PyObject_DelAttr() at the moment, > would then trigger the warning as well. > > As a result, the deprecation would have to be extended across > more releases than the usual cycle. > > A first step would be to replace the macro with a proper function > to avoid false positive warnings, even when using the correct API. > > Then we could add a warning to the PyObject_SetAttr() function and > hope that not too many projects use the stable ABI as basis to > have C extensions work across several releases. Ah, I forgot to take the stable ABI guarantee into account - you're right, it isn't possible to introduce the deprecation without making an addition to the stable ABI, which would mean extension modules relying on the stable ABI would need to be rebuilt, rather defeating the purpose of the stable ABI guarantee. I think that puts the idea squarely in "we can't do it" territory, since the benefit on offer through the deprecation process is only a much easier debugging session when someone is trying to track down the root cause of an unexpectedly missing attribute on an object. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 02.12.2015 13:29, Serhiy Storchaka wrote: > On 02.12.15 12:06, Victor Stinner wrote: >> 2015-12-02 9:42 GMT+01:00 Serhiy Storchaka : >>> You have enough time to update your projects, and you can update them >>> uniformly for all versions. And may be you will found few weird bugs related >>> to misuse of Set* API. >> >> Did you check popular projects using C extensions to check if they >> call Set*() functions to delete attributes/items? > > I have checked following projects. > > regex, simplejson, Pillow, PyQt4, LibreOffice, PyGTK, PyICU, pyOpenSSL, > libxml2, Boost, psutil, > mercurial don't use PyObject_SetAttr at all. > > NumPy, pgobject don't use PyObject_SetAttr for deleting. > > PyYAML and lxml use PyObject_SetAttr only in code generated by Cython and > never use it for deleting. While I don't think deleting attributes is a very common thing to do in any Python code base (unless you need to break circular references or explicitly want to free resources), the fact that PyObject_DelAttr() itself is implemented as macro using the NULL attribute value clearly creates an API incompatibility when removing this functionality or generating warnings, since all code using the correct PyObject_DelAttr() at the moment, would then trigger the warning as well. As a result, the deprecation would have to be extended across more releases than the usual cycle. A first step would be to replace the macro with a proper function to avoid false positive warnings, even when using the correct API. Then we could add a warning to the PyObject_SetAttr() function and hope that not too many projects use the stable ABI as basis to have C extensions work across several releases. Overall, I'm not sure whether it's worth the trouble. Documenting the feature and adding a deprecation notice to just the documentation would likely be better. We could then remove the functionality in Python 4. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Dec 02 2015) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 02.12.15 12:06, Victor Stinner wrote: 2015-12-02 9:42 GMT+01:00 Serhiy Storchaka : You have enough time to update your projects, and you can update them uniformly for all versions. And may be you will found few weird bugs related to misuse of Set* API. Did you check popular projects using C extensions to check if they call Set*() functions to delete attributes/items? I have checked following projects. regex, simplejson, Pillow, PyQt4, LibreOffice, PyGTK, PyICU, pyOpenSSL, libxml2, Boost, psutil, mercurial don't use PyObject_SetAttr at all. NumPy, pgobject don't use PyObject_SetAttr for deleting. PyYAML and lxml use PyObject_SetAttr only in code generated by Cython and never use it for deleting. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
2015-12-02 9:42 GMT+01:00 Serhiy Storchaka : > You have enough time to update your projects, and you can update them > uniformly for all versions. And may be you will found few weird bugs related > to misuse of Set* API. Did you check popular projects using C extensions to check if they call Set*() functions to delete attributes/items? If the feature is used, I suggest to document and test it, not remove it. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
середа, 02-гру-2015 08:30:35 ви написали: > Le 1 déc. 2015 16:51, "Serhiy Storchaka" a écrit : > > Wouldn't be worth to deprecate deleting with Set* functions? Neither > > other abstract Set* APIs, not concrete Set* APIs don't support deleting. > >Deleting with Set* API can be unintentional and hide a bug. > Wow wow wow, what? No, dont break Python C API for purity. 8 years later, > we are still porting projects to python 3 And we are not done yet. I suggest just to deprecate this feature. I'm not suggesting to remove it in the foreseeable future (at least before 4.0). > Practicability beats purity. I don't think this argument applies here. Two things make the deprecation more painless than usual: 1. This feature has never been documented. 2. PyObject_DelAttr() exists from the start (from the time of adding Generic Abstract Object Interface). You have enough time to update your projects, and you can update them uniformly for all versions. And may be you will found few weird bugs related to misuse of Set* API. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 2 December 2015 at 01:50, Serhiy Storchaka wrote: > On 25.11.15 08:39, Nick Coghlan wrote: >> So it looks to me like replicating the NULL-handling behaviour of the >> slots in the public Set* APIs was intentional, and it's just the >> documentation of that detail that was missed (since most folks >> presumably use the Del* convenience APIs instead). > > I'm not sure. This looks rather as implementation detail to me. There cases > found by you are the only cases in the core/stdlib that call > PyObject_SetAttr with third argument is NULL. Tests are passed after > replacing Set* functions with Del* functions in these cases and making Set* > functions to reject value is NULL. [1] Which means at the very least, folks relying on the current behaviour are relying on untested functionality, and would be better of switching to the tested APIs regardless of what happens on the deprecation front. > Wouldn't be worth to deprecate deleting with Set* functions? Neither other > abstract Set* APIs, not concrete Set* APIs don't support deleting. Deleting > with Set* API can be unintentional and hide a bug. Since the behaviour is currently neither documented not tested, and it doesn't raise any new Python 2/3 migation issues, I don't personally mind deprecating the "delete via set" APIs for 3.6 - as you say, having "set this field/attribute to this value" occasionally mean "delete this field/attribute" if a pointer is NULL offers a surprising second way to do something that already has a more explicit spelling. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 25.11.15 08:39, Nick Coghlan wrote: On 25 November 2015 at 07:33, Guido van Rossum wrote: Ooooh, that's probably really old code. I guess for the slots the reasoning is to save on slots. For the public functions, alas it will be hard to know if anyone is depending on it, even if it's undocumented. Perhaps add a deprecation warning to these if the value is NULL for one release cycle? I did a quick scan for "PyObject_SetAttr", and it turns out PyObject_DelAttr is only a convenience macro for calling PyObject_SetAttr with NULL as the value argument. bltinmodule.c and ceval.c also both include direct calls to PyObject_SetAttr with "(PyObject *)NULL" as the value argument. Investigating some of the uses that passed a variable as the value argument, one case is the weakref proxy implementation, which uses PyObject_SetAttr on the underlying object in its implementation of the setattr slot in the proxy. So it looks to me like replicating the NULL-handling behaviour of the slots in the public Set* APIs was intentional, and it's just the documentation of that detail that was missed (since most folks presumably use the Del* convenience APIs instead). I'm not sure. This looks rather as implementation detail to me. There cases found by you are the only cases in the core/stdlib that call PyObject_SetAttr with third argument is NULL. Tests are passed after replacing Set* functions with Del* functions in these cases and making Set* functions to reject value is NULL. [1] Wouldn't be worth to deprecate deleting with Set* functions? Neither other abstract Set* APIs, not concrete Set* APIs don't support deleting. Deleting with Set* API can be unintentional and hide a bug. [1] http://bugs.python.org/issue25773 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
On 25 November 2015 at 07:33, Guido van Rossum wrote: > Ooooh, that's probably really old code. I guess for the slots the > reasoning is to save on slots. For the public functions, alas it will > be hard to know if anyone is depending on it, even if it's > undocumented. Perhaps add a deprecation warning to these if the value > is NULL for one release cycle? I did a quick scan for "PyObject_SetAttr", and it turns out PyObject_DelAttr is only a convenience macro for calling PyObject_SetAttr with NULL as the value argument. bltinmodule.c and ceval.c also both include direct calls to PyObject_SetAttr with "(PyObject *)NULL" as the value argument. Investigating some of the uses that passed a variable as the value argument, one case is the weakref proxy implementation, which uses PyObject_SetAttr on the underlying object in its implementation of the setattr slot in the proxy. So it looks to me like replicating the NULL-handling behaviour of the slots in the public Set* APIs was intentional, and it's just the documentation of that detail that was missed (since most folks presumably use the Del* convenience APIs instead). Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deleting with setting C API functions
Ooooh, that's probably really old code. I guess for the slots the reasoning is to save on slots. For the public functions, alas it will be hard to know if anyone is depending on it, even if it's undocumented. Perhaps add a deprecation warning to these if the value is NULL for one release cycle? On Tue, Nov 24, 2015 at 1:21 PM, Serhiy Storchaka wrote: > Slots like PyTypeObject.tp_setattr, PySequenceMethods.sq_ass_item, > PyMappingMethods.mp_ass_subscript are used not only for setting > attribute/item value, but also for deleting attribute/item (if value is > NULL). This is not documented and should be. [1] Correspondingly public API > functions like PyObject_SetAttr, PyObject_SetItem, PySequence_SetItem, > PySequence_SetSlice, PyMapping_SetItemString can be used for deleting. But > all these functions have special counterparts for deleting: PyObject_DelAttr > etc. > > The question is wherever deleting ability of Set-functions is intentional, > should we document this or deprecate and then delete? > > [1] http://bugs.python.org/issue25701 > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Deleting with setting C API functions
Slots like PyTypeObject.tp_setattr, PySequenceMethods.sq_ass_item, PyMappingMethods.mp_ass_subscript are used not only for setting attribute/item value, but also for deleting attribute/item (if value is NULL). This is not documented and should be. [1] Correspondingly public API functions like PyObject_SetAttr, PyObject_SetItem, PySequence_SetItem, PySequence_SetSlice, PyMapping_SetItemString can be used for deleting. But all these functions have special counterparts for deleting: PyObject_DelAttr etc. The question is wherever deleting ability of Set-functions is intentional, should we document this or deprecate and then delete? [1] http://bugs.python.org/issue25701 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
