Re: [Cython] PR on refcounting memoryview buffers
Den 20. feb. 2013 kl. 11:55 skrev Sturla Molden : > > Den 18. feb. 2013 kl. 19:32 skrev Sturla Molden : > >> The problem this addresses is when GCC does not use atomic builtins and >> emits __synch_fetch_and_add_4 and __synch_fetch_and_sub_4 when Cython are >> internally refcounting memoryview buffers. For some reason it can even >> happen on x86 and amd64. > > Specifically, atomic builtins are not used when compiling for i386, which is > MinGWs default target architecture (unless we specify a different -march). > GCC will always encounter this problem when targeting i386. > > Thus the correct fix is to use fallback when GCC is targeting i386 — not when > GCC is targeting MS Windows. > > So I am closing this PR. But Mark's fix must be corrected, because it does > not really address the problem (which is i386, not MinGW)! > Please consider this pull-request: https://github.com/cython/cython/pull/190 Sturla Molden ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Py_UNICODE* string support
On Mon, 04 Mar 2013 01:56:59 +0600, Stefan Behnel wrote: As one little nit-pick, may I ask you to rename the new name references to "unicode" into "py_unicode" in your code? For example, "is_unicode", "get_unicode_const", "unicode_const_index", etc. Given that Py_UNICODE is no longer the native equivalent of Python's unicode type in Py3.3, I'd like to avoid confusion in the code. The name "unicode" is much more likely to refer to the builtin Python type than to a native C type when it appears in Cython's sources. Actually, "py_unicode" is even more likely to be mistaken for Python-level unicode. There are already pairs of methods like get_string_const (C-level) + get_py_string_const (Py-level). I suggest one of "py_unicode_ptr", "py_unicode_str", "wstring", "wide_string", "ustring", "unicode_string" to unambiguously refer to Py_UNICODE* variables and constants. Take yout pick. Oh, and yet another thing: could you write up some documentation for this in docs/src/tutorial/strings.rst ? Basically a Windows/wchar_t related section, that also warns about the inefficiency in Py3.3, so that users don't accidentally assume it's efficient for anything that needs to be portable. Sure, I'm writing the docs now. Best regards, Nikita Nemkin ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Py_UNICODE* string support
Nikita Nemkin, 04.03.2013 18:39: > On Mon, 04 Mar 2013 01:56:59 +0600, Stefan Behnel wrote: >> As one little nit-pick, may I ask you to rename the new name references >> to "unicode" into "py_unicode" in your code? For example, "is_unicode", >> "get_unicode_const", "unicode_const_index", etc. Given that Py_UNICODE is >> no longer the native equivalent of Python's unicode type in Py3.3, I'd >> like to avoid confusion in the code. The name "unicode" is much more >> likely to >> refer to the builtin Python type than to a native C type when it appears >> in Cython's sources. > > Actually, "py_unicode" is even more likely to be mistaken for Python-level > unicode. There are already pairs of methods like > get_string_const (C-level) + get_py_string_const (Py-level). Agreed. > I suggest one of "py_unicode_ptr", "py_unicode_str", "wstring", "wide_string", > "ustring", "unicode_string" to unambiguously refer to Py_UNICODE* variables > and constants. Take yout pick. I think "pyunicode_ptr" or even just "pyunicode" makes it quite clear what it's about and specifically that "pyunicode" is actually a type name, not a "py_something". Even "pyunicode_array" would work, although it might suggest that we know more at compile time than we do, such as the length. I'll let you choose between these three, although I'm leaning slightly towards an order of preference as they appear above. >> Oh, and yet another thing: could you write up some documentation for this >> in docs/src/tutorial/strings.rst ? Basically a Windows/wchar_t related >> section, that also warns about the inefficiency in Py3.3, so that users >> don't accidentally assume it's efficient for anything that needs to be >> portable. > > Sure, I'm writing the docs now. Nice. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] nonecheck and as_none_safe_node method
In ExprNodes.py there are several places where method `as_none_safe_node` was applied in order to wrap a node by NoneCheckNode. I think it would be resonable to apply that mostly only in cases when noncheck=True. Here are possible changes in ExprNodes.py: https://github.com/intellimath/cython/commit/bd041680b78067007ad6b9894a2f2c18514e397c Zaur Shibzukhov ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] nonecheck and as_none_safe_node method
2013/3/5 Zaur Shibzukhov > In ExprNodes.py there are several places where method `as_none_safe_node` > was applied in order to wrap a node by NoneCheckNode. > I think it would be resonable to apply that mostly only in cases when > noncheck=True. > > Here are possible changes in ExprNodes.py: > > https://github.com/intellimath/cython/commit/bd041680b78067007ad6b9894a2f2c18514e397c > > This change would prevent generation of None checking of an objects (lists, tuples, unicode) when nonecheck=True. Any adeas? Zaur Shibzukhov ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] nonecheck and as_none_safe_node method
2013/3/5 Zaur Shibzukhov > 2013/3/5 Zaur Shibzukhov > >> In ExprNodes.py there are several places where method `as_none_safe_node` >> was applied in order to wrap a node by NoneCheckNode. >> I think it would be resonable to apply that mostly only in cases when >> noncheck=True. >> >> Here are possible changes in ExprNodes.py: >> >> https://github.com/intellimath/cython/commit/bd041680b78067007ad6b9894a2f2c18514e397c >> >> This change would prevent generation of None checking of an objects > (lists, tuples, unicode) when nonecheck=True. > Sorry... when nonecheck=False > Any adeas? > > Zaur Shibzukhov -- С уважением, Шибзухов З.М. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] nonecheck and as_none_safe_node method
Zaur Shibzukhov, 05.03.2013 07:21: > In ExprNodes.py there are several places where method `as_none_safe_node` > was applied in order to wrap a node by NoneCheckNode. > I think it would be resonable to apply that mostly only in cases when > noncheck=True. > > Here are possible changes in ExprNodes.py: > https://github.com/intellimath/cython/commit/bd041680b78067007ad6b9894a2f2c18514e397c I consider the nonecheck option a quirk. In many, many cases, it's not obvious to a user to what constructs it applies. For example, we use it to guard against crashes when we optimise code, e.g. by inlining parts of a C-API function, when iterating over builtins, etc. In most of these cases, it depends on more than one parameter if the optimised code will be applied (and thus no None check) or the fallback, which usually does its own complete set of safety checks. So it's one of those options that may work safely in all unit tests and then crash in production. Remember, most cases where we leave a None check in the code are not those where it's obvious that a variable cannot be None because it was just assigned a non-None value. Most cases are about function arguments, i.e. values that come from outside of the current function, and thus are not "obviously" correct even for the human reader or author of the code. Also, I'm yet to see a case where a None check really makes a difference in performance. Often enough, the C compiler will be able to move them out of loops or drop them completely because it already saw a None check against the same local variable earlier on. In those cases, it's just Cython not being smart enough to drop them itself, but without an impact on runtime performance. And even if the C compiler is not smart enough either, at least the branch prediction of the processor will strike in the relevant cases (i.e. inside of loops) and reduce the overhead to "pretty much zero". All of this makes em think that we should be very careful when we consider this option for the generated code. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] nonecheck and as_none_safe_node method
2013/3/5 Stefan Behnel > Zaur Shibzukhov, 05.03.2013 07:21: > > In ExprNodes.py there are several places where method `as_none_safe_node` > > was applied in order to wrap a node by NoneCheckNode. > > I think it would be resonable to apply that mostly only in cases when > > noncheck=True. > > > > Here are possible changes in ExprNodes.py: > > > https://github.com/intellimath/cython/commit/bd041680b78067007ad6b9894a2f2c18514e397c > > I consider the nonecheck option a quirk. In many, many cases, it's not > obvious to a user to what constructs it applies. For example, we use it to > guard against crashes when we optimise code, e.g. by inlining parts of a > C-API function, when iterating over builtins, etc. In most of these cases, > it depends on more than one parameter if the optimised code will be applied > (and thus no None check) or the fallback, which usually does its own > complete set of safety checks. So it's one of those options that may work > safely in all unit tests and then crash in production. > > Remember, most cases where we leave a None check in the code are not those > where it's obvious that a variable cannot be None because it was just > assigned a non-None value. Most cases are about function arguments, i.e. > values that come from outside of the current function, and thus are not > "obviously" correct even for the human reader or author of the code. > > Also, I'm yet to see a case where a None check really makes a difference in > performance. Often enough, the C compiler will be able to move them out of > loops or drop them completely because it already saw a None check against > the same local variable earlier on. In those cases, it's just Cython not > being smart enough to drop them itself, but without an impact on runtime > performance. And even if the C compiler is not smart enough either, at > least the branch prediction of the processor will strike in the relevant > cases (i.e. inside of loops) and reduce the overhead to "pretty much zero". > > All of this makes em think that we should be very careful when we consider > this option for the generated code. > > I agree that directive nonecheck=False is dangarous in general. This change mainly affect builtin objects (lists, tuples, dicts, unicode) and some situation in function/method calls. And it assume that when you apply this directive you know what you are doing and why. Note that Cython now already set nonecheck=False (but boundcheck and wraparound sets to True in Options.py) by default. But now it not affect builtin types and some special cases. May be the safer strategy is to set nonecheck=True by default and allow locally to apply nonecheck(False) when developer believes that it is necessary? ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
