[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: "The **exact** regression is that I could run the test suite without any crash or freeze on this AIX system with 3.7.6 and I cannot in 3.7.7. At the very least this is due to the fact that there is a new test that crashes/hangs." In other words, contrary to your earlier dismissal, you did NOT run _decimal on AIX with MAX_PREC but just ran the 3.7.6 tests that do not include any tests with MAX_PREC. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: It is also instructive how Granlund views xlc: https://gmplib.org/list-archives/gmp-bugs/2010-November/002119.html -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: A focused issue report would include at least the following: 1) Acknowledge that gcc builds work on the AIX buildbots (a fact that has been entirely ignored so far). 2) State the exact regression: msg364373 (which was also ignored, are you using xlc or xlc_r?) says that there have always been problems with xlc unless configured in a very specific manner. Then obviously we need the exact Python code that worked in 3.7.6 but not in 3.7.7. In short, given the flakiness of the xlc toolchain I'm not even sure if anything can be classified as a regression. Most xlc users I've talked to have switched to gcc. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Change by Stefan Krah : -- assignee: -> skrah keywords: -3.7regression, patch ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Change by Stefan Krah : -- assignee: -> skrah ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: BTW, if you are compiling with xlc and there"s no xlc buildbot, perhaps a company-that-shall-not-be-named should provide one. I mean, it's not okay to complain about a regression and then mention xlc about 10 mails later. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: These flags worked for xlc when snakebite was still up: ./configure CC=xlc_r AR="ar -X64" CFLAGS="-q64 -qmaxmem=7" LDFLAGS="-q64" -qmaxmem was always finicky, I remember segfaults too (AIX problem, not libmpdec problem). -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: Hi Michael, in case you have built 3.7.7 on AIX, have you observed any problems with test_decimal? -- nosy: +Michael.Felt ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: > Well, I just did and I can confirm that reverting the 3.7 backport fixes the > problem. If you are fortunate enough to have access to an AIX system, I guess you have to find out why POWER6 AIX 3.8 and PPC64 AIX 3.8 apparently work on https://buildbot.python.org/ but your 3.7 does not. It seems rather strange to me. Are you compiling --with-system-libmpdec? -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: > This code has introduced a regression in AIX in Python 3.7.7 Also this is a rather bold statement since probably no one has ever run _decimal on AIX with MAX_PREC. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: > This looks like a "new feature/improvement". Why was this code backported to > a stable version? Thanks for the lecture. This is an esoteric case between bugfix and feature that only occurs with very large context precisions. If Bloomberg isn't happy with _decimal, they can stop using it. I'm not paid by them. Which AIX buildbot fails? There are many green ones. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Change by Stefan Krah : -- pull_requests: +18317 pull_request: https://github.com/python/cpython/pull/18969 ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: memoryview only supports the native format, so I've disabled the (wrong) test that casts arrays with arbitrary values to _Bool. So memoryview is done. IMO the problem in _struct is that it swaps the x->unpack function for the native one, which does not seem right for _Bool: /* Scan through the native table, find a matching entry in the endian table and swap in the native implementations whenever possible (64-bit platforms may not have "standard" sizes) */ If one disables that swap, the tests pass here. -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: New changeset f8ce3e2dae277baa2ef92e8a3e935953dc6c3f39 by Miss Islington (bot) in branch '3.8': bpo-39689: Do not test undefined casts to _Bool (GH-18964) (#18966) https://github.com/python/cpython/commit/f8ce3e2dae277baa2ef92e8a3e935953dc6c3f39 -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: New changeset 636eecc16229432ec8e26e6da287c52f3ca3 by Miss Islington (bot) in branch '3.7': bpo-39689: Do not test undefined casts to _Bool (GH-18964) (#18965) https://github.com/python/cpython/commit/636eecc16229432ec8e26e6da287c52f3ca3 -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: New changeset 1ae9cde4b2323235b5f9ff4bc76e4175a2257172 by Stefan Krah in branch 'master': bpo-39689: Do not test undefined casts to _Bool (GH-18964) https://github.com/python/cpython/commit/1ae9cde4b2323235b5f9ff4bc76e4175a2257172 -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Change by Stefan Krah : -- pull_requests: +18312 pull_request: https://github.com/python/cpython/pull/18964 ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: The memcpy() is NOT a hack and performs exactly the same operation as casting the pointer and then dereferencing, only in a manner that avoids unaligned accesses. On platforms like x86 the memcpy() is optimized to a simple assignment. Casting the pointer and then dereferencing would also be subject to the UB warnings. You are the one who wanted to *introduce* a hack by dereferencing as char and then cast to _Bool. :-) -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment:
The docs for native mode (we now always assume C99):
"The '?' conversion code corresponds to the _Bool type defined by C99."
The memoryview tests that fail are essentially auto-generated and not
prescriptive. They just happened to work without UBSan:
>>> x = array.array("B", [0,1,2,3])
>>> m = memoryview(x)
>>> m.format
'B'
>>> c = m.cast("?") # Wrong!
>>> c.tolist()
[False, True, True, True]
I don't see the point in working around UB for the purposes of
displaying the wrong cast.
If you do subsequent array operations with the variable "c", UB will happen
there.
In theory you can do the same with a cast from unsigned to signed
integers. Signed integers are allowed to be trap representations,
though I don't know an actual platform where this is the case.
--
___
Python tracker
<https://bugs.python.org/issue39689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: I checked that NumPy also packs correctly: >>> import numpy as np >>> x = np.array([0,1,2,3], dtype=np.bool) >>> x.tobytes() b'\x00\x01\x01\x01' So I vote for not handling incorrectly packed values and removing "and any non-zero value will be True when unpacking" from the docs, which does not seem to make any sense for _Bool. memoryview also does not guard against the theoretical possibility of incorrectly packed values being trap representations. Values need to be packed correctly, or you get UB. -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: Okay, in memoryview the cast tests can trigger UB checks. memoryview assumes that bool is packed correctly, so just casting does not work. Casting anything to bool is of course a bit silly anyway. diff --git a/Lib/test/test_buffer.py b/Lib/test/test_buffer.py index 6178ffde7a..86cf4c309f 100644 --- a/Lib/test/test_buffer.py +++ b/Lib/test/test_buffer.py @@ -2758,6 +2758,8 @@ class TestBufferProtocol(unittest.TestCase): tsize = struct.calcsize(tfmt) n = prod(_tshape) * tsize obj = 'memoryview' if is_byte_format(tfmt) else 'bytefmt' +if "?" in tfmt: +continue for fmt, items, _ in iter_format(n, obj): size = struct.calcsize(fmt) shape = [n] if n > 0 else [] -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment: So which memoryview test (unless it is using the struct module) relies on UB? -- ___ Python tracker <https://bugs.python.org/issue39689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)
Stefan Krah added the comment:
Concerning memoryview, I've looked at this very briefly: memoryview
does not pack values directly, it casts first, which is legal:
#define PACK_SINGLE(ptr, src, type) \
do { \
type x; \
x = (type)src; \
memcpy(ptr, (char *), sizeof x); \
} while (0)
This macro has exposed compiler bugs before.
--
nosy: +skrah
___
Python tracker
<https://bugs.python.org/issue39689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Stefan Krah added the comment: Thanks Ned, closing then. Evgeny, please reopen if you see it again (I ran the tests for about 20 min, way above the usual reproduction time of 1 min). Thanks for the very instructive test case! -- status: open -> closed ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Change by Stefan Krah : -- resolution: -> fixed ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Change by Stefan Krah : -- stage: patch review -> resolved ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: #39776 is fixed, but _pydecimal was also affected, see msg363266. -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Stefan Krah added the comment: Setting to release blocker, but please move to deferred again if a release is almost finished. -- nosy: +lukasz.langa, ned.deily priority: deferred blocker -> release blocker ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Stefan Krah added the comment:
Also _pydecimal was affected. This is a modified version of Evgeny's
test case:
from _pydecimal import *
from time import sleep
from random import randint
import sys
sys.setswitchinterval(0.001)
def usleep(x):
sleep(x/100.0)
class Test:
def __init__(self, threadno):
self.threadno = threadno
def GetCallback(self):
usleep(randint(0, 9));
setcontext(Context(Emax=self.threadno))
usleep(randint(0, 9))
c = getcontext()
try:
if c.Emax != self.threadno:
raise RuntimeError("GetCallback: another thread changed this
thread's context")
except AttributeError:
raise RuntimeError("GetCallback: type(c)==%s and c.Emax
disappeared" % type(c))
usleep(randint(0, 9))
return self.Callback
def Callback(self):
c = getcontext()
try:
c.Emax = self.threadno
except AttributeError:
raise RuntimeError("Callback: type(c)==%s and c.Emax disappeared" %
type(c))
def DoTest(threadno):
return Test(threadno).GetCallback()
It produces one of the exceptions or a segfault. You may have
to build in release mode to get the race.
--
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Change by Stefan Krah : Added file: https://bugs.python.org/file48945/pydecimal_cases.zip ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Stefan Krah added the comment: Since many duplicate tstate ids are generated in the test case before the crash happens, I guess a set of tstates with the same id silently uses the cached context of the "winner" tstate. This can lead to incorrect results without noticing. -- priority: critical -> deferred blocker ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Change by Stefan Krah : -- assignee: -> skrah ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Change by Stefan Krah : -- title: Crash in decimal module in heavy-multithreaded scenario -> PyContextVar_Get(): crash due to race condition in updating tstate->id ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: New changeset 852aee69f49c654a03ad1f64d90a78ba8848e2c6 by Stefan Krah in branch '3.7': bpo-39776: Lock ++interp->tstate_next_unique_id (GH-18746) https://github.com/python/cpython/commit/852aee69f49c654a03ad1f64d90a78ba8848e2c6 -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: New changeset 5a92f42d8723ee865be80f028d402204649da15d by Stefan Krah in branch '3.8': bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746) (#18752) https://github.com/python/cpython/commit/5a92f42d8723ee865be80f028d402204649da15d -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Change by Stefan Krah : -- pull_requests: +18108 pull_request: https://github.com/python/cpython/pull/18753 ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Change by Stefan Krah : -- pull_requests: +18107 pull_request: https://github.com/python/cpython/pull/18752 ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: New changeset b3b9ade4a3d3fe00d933bcd8fc5c5c755d1024f9 by Stefan Krah in branch 'master': bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746) https://github.com/python/cpython/commit/b3b9ade4a3d3fe00d933bcd8fc5c5c755d1024f9 -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: I think the PR fixes the issue but I have to run longer tests still. Threads created by PyGILState_Ensure() could have a duplicate tstate->id, which confused the ContextVar caching machinery. -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Change by Stefan Krah : -- keywords: +patch pull_requests: +18101 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/18746 ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: To make things more clear (it is good that Gregory asked): ONLY use this flag if you don't use coroutines at all OR control ALL async libraries in your application. Random async libraries from PyPI may rely on the ContextVar and silently give incorrect results. -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: > If someone builds an interpreter with this configure flag, does it break > compatibility with anything that code may have started to expect as of 3.7? Yes, anything that relies on the implicit context being coroutine-safe does not work. The only test that expectedly breaks in the stdlib is: Lib/test/test_asyncio/test_context.py Basically you can't use operators inside coroutines in the same thread if both coroutines use a different implicit context. You can of course replace all operators inside coroutines by the corresponding context methods: # Unsafe with TLS context: async def foo(): with localcontext(Context(prec=10)): x = Decimal(1) / 9 # Safe, just avoid the TLS context: async def foo(): c = Context(prec=10) x = c.divide(Decimal(1), 9) -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: For the people who find this issue via a search engine: 1) The defaults in 3.7, 3.8 and 3.9 are unchanged (you get the ContextVar). 2) ./configure --without-decimal-contextvar enables the exact TLS context code from 3.3-3.6. There is no new code. 3) On Windows you need to edit PC\pyconfig.h to enable the TLS context. 4) There are extensive tests for the flag in: Modules/_decimal/tests/runall-memorydebugger.sh -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Change by Stefan Krah : -- nosy: +boytsovea ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: New changeset c4ca1f8f24118dc5c29e16118fb35a13963af290 by Stefan Krah in branch '3.7': [3.7] bpo-39794: Add --without-decimal-contextvar (GH-18702) https://github.com/python/cpython/commit/c4ca1f8f24118dc5c29e16118fb35a13963af290 -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: New changeset 4d7012410cf4f91cbca4c406f4747289c2802333 by Stefan Krah in branch '3.8': [3.8] bpo-39794: Add --without-decimal-contextvar (GH-18702) https://github.com/python/cpython/commit/4d7012410cf4f91cbca4c406f4747289c2802333 -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Change by Stefan Krah : -- pull_requests: +18074 pull_request: https://github.com/python/cpython/pull/18714 ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Change by Stefan Krah : -- pull_requests: +18072 pull_request: https://github.com/python/cpython/pull/18713 ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: New changeset 815280eb160af637e1347213659f9236adf78f80 by Stefan Krah in branch 'master': bpo-39794: Add --without-decimal-contextvar (#18702) https://github.com/python/cpython/commit/815280eb160af637e1347213659f9236adf78f80 -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Change by Stefan Krah : -- keywords: +patch pull_requests: +18064 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/18702 ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Stefan Krah added the comment: Also, when I'm debugging things like #39776, I don't want to switch between Python versions. -- ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
Change by Stefan Krah : -- assignee: -> skrah components: +Extension Modules stage: -> needs patch type: -> behavior versions: +Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39794] Add --without-decimal-contextvar option to use just threads in decimal
New submission from Stefan Krah : #39776 has shown that it is hard to understand the interaction between ContextVars and threading in embedded scenarios. I want to understand the code again, so I'm adding back a compile time option to enable the thread local context that was present prior to f13f12d8d. -- messages: 362971 nosy: mark.dickinson, rhettinger, skrah priority: normal severity: normal status: open title: Add --without-decimal-contextvar option to use just threads in decimal ___ Python tracker <https://bugs.python.org/issue39794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: > With python 3.7.3 without https://github.com/python/cpython/pull/5278 works > just fine. Thanks, I'm now getting the same results as you. Looking at the smaller test case, I also agree that it should work (as it did in 3.6). -- keywords: +3.7regression resolution: not a bug -> stage: -> needs patch type: -> crash ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: Regarding *my* issue, it could be anything, e.g. a missing call to PyEval_InitThreads() in 3.6: "Changed in version 3.7: This function is now called by Py_Initialize(), so you don’t have to call it yourself anymore." This is why we need to eliminate pybind11 so we can see what is actually going on. -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Change by Stefan Krah : -- resolution: not a bug -> stage: resolved -> ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: Note that my pybind11 is from GitHub master, it can also be a pybind11 issue. It is interesting that you cannot reproduce your original issue with 3.6, so I'm reopening this issue. I think we need a reproducer without pybind11 though, could you tweak Programs/_testembed.c (from the CPython sources) to run the crash script? -- status: closed -> open ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: This is 3.6.7, compiled --with-pydebug: $ ./main Aborted (core dumped) (gdb) bt #0 0x7f9974077428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x7f997407902a in __GI_abort () at abort.c:89 #2 0x0056e2d1 in Py_FatalError (msg=msg@entry=0x62ccf8 "Python memory allocator called without holding the GIL") at Python/pylifecycle.c:1462 #3 0x004c0cec in _PyMem_DebugCheckGIL () at Objects/obmalloc.c:1963 #4 0x004c0d27 in _PyMem_DebugMalloc (ctx=0x8f7220 <_PyMem_Debug+96>, nbytes=75) at Objects/obmalloc.c:1971 #5 0x004c204e in PyObject_Malloc (size=) at Objects/obmalloc.c:479 #6 0x004ec12f in PyUnicode_New (size=10, maxchar=) at Objects/unicodeobject.c:1281 #7 0x005162f4 in _PyUnicodeWriter_PrepareInternal (writer=writer@entry=0x7f9971ca4cf0, length=length@entry=10, maxchar=, maxchar@entry=127) at Objects/unicodeobject.c:13565 #8 0x0051af20 in PyUnicode_DecodeUTF8Stateful (s=0x61d15b "crash_test", size=10, errors=errors@entry=0x0, consumed=consumed@entry=0x0) at Objects/unicodeobject.c:5067 #9 0x0051c6b0 in PyUnicode_FromString (u=) at Objects/unicodeobject.c:2077 #10 0x00563c1c in PyImport_ImportModule (name=) at Python/import.c:1266 #11 0x004531dd in pybind11::module::import (name=0x61d15b "crash_test") at ./pybind11/include/pybind11/pybind11.h:849 #12 0x00446434 in ThreadFunc () at main.cpp:30 #13 0x0046a1b1 in std::_Bind_simple::_M_invoke<>(std::_Index_tuple<>) (this=0x10c28d8) at /usr/include/c++/5/functional:1531 #14 0x0046a10a in std::_Bind_simple::operator()() (this=0x10c28d8) at /usr/include/c++/5/functional:1520 #15 0x0046a09a in std::thread::_Impl >::_M_run() (this=0x10c28c0) at /usr/include/c++/5/thread:115 #16 0x7f99749e3c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #17 0x7f99750bb6ba in start_thread (arg=0x7f9971ca5700) at pthread_create.c:333 #18 0x7f997414941d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: I built your example with 3.6: git clone https://github.com/pybind/pybind11 wget https://bugs.python.org/file48923/decimal_crash.zip unzip decimal_crash.zip git checkout v3.6.7 ./configure --with-pydebug make g++ -std=c++11 -pthread -Wno-unused-result -Wsign-compare -g -Og -Wall -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I. -I./Include -I./pybind11/include -c main.cpp g++ -pthread -Xlinker -export-dynamic -o main main.o libpython3.6dm.a -lpthread -ldl -lutil -lm cp python python3 PATH=.:$PATH ./main And I literally get this error (not always, it may take 10 runs or so): $ ./main Fatal Python error: Python memory allocator called without holding the GIL Thread 0x7f1e73fff700 (most recent call first): Thread 0x7f1e7b836700 (most recent call first): Thread 0x7f1e7a834700 (most recent call first): Thread 0x7f1e7b035700 (most recent call first): Thread 0x7f1e7d039700 (most recent call first): Thread 0x7f1e7c838700 (most recent call first): Current thread 0x7f1e7c037700 (most recent call first): Thread 0x7f1e7e84f740 (most recent call first): Aborted (core dumped) So no, I don't think the GIL handling is correct. -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Change by Stefan Krah : -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: I've briefly looked at the zip archive. Without going much into the C++ module as a whole, this should not be done: gil_unlocker.UnlockGILAndSleep() self.val = decimal.Decimal(1) / decimal.Decimal(7) gil_unlocker.UnlockGILAndSleep() If you want C++ threads with a released GIL, you should use libmpdec directly and not the Python module. -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Stefan Krah added the comment: Before I look at the example code: Can you also reproduce this with Python 3.6? The threading code in _decimal was changed to a ContextVar in 3.7. There's a high chance though that the problem is in the c++ module. -- nosy: +skrah ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39704] Disable code coverage
Stefan Krah added the comment: For me even a mail with a single line would be too much. I can filter that in my mail client but not on GitHub. Speaking about that, I also don't want to get mail from Bevedere stating that I, in fact, have signed a CLA any time I open a PR. -- ___ Python tracker <https://bugs.python.org/issue39704> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset c6ecd9c14081a787959e13df33e250102a658154 by Miss Islington (bot) in branch '3.8': bpo-39576: Clarify the word size for the 32-bit build. (GH-18616) (#18618) https://github.com/python/cpython/commit/c6ecd9c14081a787959e13df33e250102a658154 -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset 24c570bbb82a7cb70576c253a73390accfa7ed78 by Miss Islington (bot) in branch '3.7': bpo-39576: Clarify the word size for the 32-bit build. (GH-18616) (#18617) https://github.com/python/cpython/commit/24c570bbb82a7cb70576c253a73390accfa7ed78 -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset b76518d43fb82ed9e5d27025d18c90a23d525c90 by Stefan Krah in branch 'master': bpo-39576: Clarify the word size for the 32-bit build. (#18616) https://github.com/python/cpython/commit/b76518d43fb82ed9e5d27025d18c90a23d525c90 -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Change by Stefan Krah : -- pull_requests: +17981 pull_request: https://github.com/python/cpython/pull/18616 ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Change by Stefan Krah : -- stage: patch review -> resolved ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: libmpdec and the docs are done, the question remains what to do with decimal.py. It has the same behavior, but I don't think users are running decimal.py with very large precisions. Anyway, unassigning myself in case anyone else wants to work on a patch. -- assignee: skrah -> resolution: -> fixed versions: +Python 3.7 ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset d6965ff026f35498e554bc964ef2be8f4d80eb7f by Miss Islington (bot) in branch '3.8': bpo-39576: docs: set context for decimal arbitrary precision arithmetic (GH-18594) (#18597) https://github.com/python/cpython/commit/d6965ff026f35498e554bc964ef2be8f4d80eb7f -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset 00e45877e33d32bb61aa13a2033e3bba370bda4d by Miss Islington (bot) in branch '3.7': bpo-39576: docs: set context for decimal arbitrary precision arithmetic (GH-18594) (#18596) https://github.com/python/cpython/commit/00e45877e33d32bb61aa13a2033e3bba370bda4d -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset a025d4ca99fb4c652465368e0b4eb03cf4b316b9 by Stefan Krah in branch 'master': bpo-39576: docs: set context for decimal arbitrary precision arithmetic (#18594) https://github.com/python/cpython/commit/a025d4ca99fb4c652465368e0b4eb03cf4b316b9 -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39704] Disable code coverage
Stefan Krah added the comment: They are allowed failures but the build is still marked in red: https://github.com/python/cpython/pull/18567 So if you look at the front page you have to click through red results only to find that the reason is code coverage. -- ___ Python tracker <https://bugs.python.org/issue39704> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: Updated docs: https://github.com/python/cpython/pull/18594 The PR uses some of Tim's suggestions while also explaining how to calculate the amount of memory used in a single large decimal. Hopefully it isn't too much information. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Change by Stefan Krah : -- pull_requests: +17961 pull_request: https://github.com/python/cpython/pull/18594 ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: "So non-integer powers are left out" in isolation would indeed be wrong, but actual sentence is unambiguously qualified with: "... since _decimal has no notion of exact non-integer powers yet.", which clearly states that exact non-integer powers exist and _decimal does not recognize them (it usually produces exact results in the respective cases but still sets Inexact). -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: Vedran, msg362365 is meant to say: "This PR implements $SOMEWHAT_MATHEMATICAL_SPEC except for inexact power." Had I put the caveat inside the statement as well, the message would have been: "This PR implements $SOMEWHAT_MATHEMATICAL_SPEC_EXCEPT_FOR_INEXACT_POWER except for inexact power." Bur GNU is not UNIX and all that... ;) -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39704] Disable code coverage
Stefan Krah added the comment: I'd definitely disable the automatic comment and prefer that the build happens on buildbot.python.org rather than affecting the GitHub build status. -- ___ Python tracker <https://bugs.python.org/issue39704> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset b6271025c640c228505dc9f194362a0c2ab81c61 by Miss Islington (bot) in branch '3.8': bpo-39576: Prevent memory error for overly optimistic precisions (GH-18581) (#18584) https://github.com/python/cpython/commit/b6271025c640c228505dc9f194362a0c2ab81c61 -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset c6f95543b4832c3f0170179da39bcf99b40a7aa8 by Miss Islington (bot) in branch '3.7': bpo-39576: Prevent memory error for overly optimistic precisions (GH-18581) (#18585) https://github.com/python/cpython/commit/c6f95543b4832c3f0170179da39bcf99b40a7aa8 -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: New changeset 90930e65455f60216f09d175586139242dbba260 by Stefan Krah in branch 'master': bpo-39576: Prevent memory error for overly optimistic precisions (GH-18581) https://github.com/python/cpython/commit/90930e65455f60216f09d175586139242dbba260 -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39704] Disable code coverage
New submission from Stefan Krah : The automated code coverage on GitHub is quite inaccurate and needlessly flags PRs as red. I'd prefer to make this opt-in. -- messages: 362367 nosy: skrah priority: normal severity: normal status: open title: Disable code coverage ___ Python tracker <https://bugs.python.org/issue39704> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: BTW, this PR implements the invariant: "If there exists an exact result at a lower precision, this result should also be returned at MAX_PREC (without MemoryError)". So non-integer powers are left out, since _decimal has no notion of exact non-integer powers yet. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: Fortunately libmpdec raises MemoryError almost instantaneously, so the PR retries the affected operations with estimated upper bounds for exact results without slowing down the common case. The docs still need updating because people will still wonder why 1 / Decimal(3) freezes the Linux machine at MAX_PREC. ;) The Python implementation cannot use this approach because memory grows slowly. I'm not sure though if anyone _would_ run _pydecimal at MAX_PREC. -- type: -> enhancement ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Change by Stefan Krah : -- keywords: +patch pull_requests: +17951 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18581 ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39701] Azure Pipelines PR broken
Change by Stefan Krah : -- status: closed -> open ___ Python tracker <https://bugs.python.org/issue39701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39701] Azure Pipelines PR broken
Stefan Krah added the comment: Re-opening this issue, Azure fails too often: https://github.com/python/cpython/pull/18577 https://dev.azure.com/Python/cpython/_build/results?buildId=58220=logs=c83831cd-3752-5cc7-2f01-8276919eb334 -- ___ Python tracker <https://bugs.python.org/issue39701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39701] Azure Pipelines PR broken
Stefan Krah added the comment: Closing and reopening the PR helped. -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue39701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39701] Azure Pipelines PR broken
New submission from Stefan Krah : There is no status report, no details link and manually committing is prohibited: https://github.com/python/cpython/pull/18569 -- ___ Python tracker <https://bugs.python.org/issue39701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39701] Azure Pipelines PR broken
Change by Stefan Krah : -- nosy: skrah priority: normal severity: normal status: open title: Azure Pipelines PR broken ___ Python tracker <https://bugs.python.org/issue39701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39610] memoryview.__len__ should raise an exception for 0d buffers
Stefan Krah added the comment: The change looks reasonable, but unfortunately this is a long-standing behavior that originates from before the Python-3.3 memoryview rewrite. It is also present in 2.7 (the previous implementation) and documented in 3.3: https://docs.python.org/3/library/stdtypes.html#memoryview I think I documented it because it looked strange back then, too. So I've to think a bit what will be affected and how this behavior came into place. -- ___ Python tracker <https://bugs.python.org/issue39610> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39552] shell scripts use legacy
New submission from Stefan Krah : As mentioned in the PR, I don't see sufficient evidence that backticks are legacy. So I'll close this. -- assignee: -> skrah nosy: +skrah resolution: -> rejected stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue39552> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: > This isn't purely academic. The `decimal` docs, at the end: Yes, that is a good point. I think for libmpdec I'll just do a trial divmod if prec > BIGNUM_THRESHOLD. > Perhaps the only thing to be done is to add words to the part of the docs > _recommending_ MAX_PREC, warning about some "unintended consequences" of > doing so. This, too. Probably some formula based on the amount of available RAM would do. -- assignee: -> skrah ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment:
The feature would be nice to have; however, if you choose the precision to
match the amount of available RAM things work (I have 8GB here, one word in the
coefficient has 19 digits for the 4 bit version):
>>> from decimal import *
>>> c = getcontext()
>>> c.prec = 8 * 2**30 // 64 * 19
>>> c.prec
2550136832
>>> i = Decimal(4)
>>> i / 2
Decimal('2')
So I wonder if we really need to do something here.
--
___
Python tracker
<https://bugs.python.org/issue39576>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: MAX_PREC is chosen so that 5*MAX_PREC does not overflow 32-bit or 64-bit signed integers. This eliminates many overflow checks for the exponent. Updating the exponent is (perhaps surprisingly) quite performance sensitive, that's why the 32-bit build does not use a 64-bit exponent. -- ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39576] Surprising MemoryError in `decimal` with MAX_PREC
Stefan Krah added the comment: With _pydecimal the memory also grows very slowly (I didn't have the patience to wait for MemoryError). I'm pretty sure decNumber also does the same, it's just easier to implement and does not slow down division for small numbers. libmpdec always favors precisions from roughly 9-34 digits whenever there's a potential performance issue. The only thing I could imagine is to special-case, say, prec > 1. -- nosy: +skrah ___ Python tracker <https://bugs.python.org/issue39576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19692] Rename Py_SAFE_DOWNCAST
Stefan Krah added the comment: Why would one not abort() in release mode? If the cast is inexact, the results will usually be so wrong that even on a web server a hard exit is preferable. I don't think the check costs much time with branch prediction. -- nosy: +skrah ___ Python tracker <https://bugs.python.org/issue19692> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37415] Error build Python with Intel compiler: doesn't provide atomic_uintptr_t
Stefan Krah added the comment: Is it available with -std=c11? It is a bit strange that we use -std=c99. I thought that header is C11: https://en.cppreference.com/w/c/atomic -- nosy: +skrah ___ Python tracker <https://bugs.python.org/issue37415> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37483] Add PyObject_CallOneArg()
Stefan Krah added the comment: > Or do you care about *using* API with underscore? If so, I'm OK to stop > changing some callers which are not tightly coupled with Python. I care about this one. Indeed I think underscore functions should be used in strategic places inside the core interpreter and not sprinkled across the whole code base by default. But in general I also care about not having too many new functions to avoid churn. -- ___ Python tracker <https://bugs.python.org/issue37483> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37483] Add PyObject_CallOneArg()
Stefan Krah added the comment: It adds yet another special case underscore function that one cannot use in external projects. So I would not say that is simpler. Has there been any performance measurement at all? -- ___ Python tracker <https://bugs.python.org/issue37483> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
