Stefan Krah added the comment:
"The **exact** regression is that I could run the test suite without any crash
or freeze on this AIX system
with 3.7.6 and I cannot in 3.7.7. At the very least this is due to the fact
that there is a new test that crashes/hangs."
In other words
Stefan Krah added the comment:
It is also instructive how Granlund views xlc:
https://gmplib.org/list-archives/gmp-bugs/2010-November/002119.html
--
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
A focused issue report would include at least the following:
1) Acknowledge that gcc builds work on the AIX buildbots (a
fact that has been entirely ignored so far).
2) State the exact regression: msg364373 (which was also ignored,
are you using
Change by Stefan Krah :
--
assignee: -> skrah
keywords: -3.7regression, patch
___
Python tracker
<https://bugs.python.org/issue39576>
___
___
Python-bugs-lis
Change by Stefan Krah :
--
assignee: -> skrah
___
Python tracker
<https://bugs.python.org/issue39576>
___
___
Python-bugs-list mailing list
Unsubscrib
Stefan Krah added the comment:
BTW, if you are compiling with xlc and there"s no xlc buildbot,
perhaps a company-that-shall-not-be-named should provide one.
I mean, it's not okay to complain about a regression and then
mention xlc about 10 mails
Stefan Krah added the comment:
These flags worked for xlc when snakebite was still up:
./configure CC=xlc_r AR="ar -X64" CFLAGS="-q64 -qmaxmem=7" LDFLAGS="-q64"
-qmaxmem was always finicky, I remember segfaults too (AIX
Stefan Krah added the comment:
Hi Michael, in case you have built 3.7.7 on AIX, have you observed
any problems with test_decimal?
--
nosy: +Michael.Felt
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
> Well, I just did and I can confirm that reverting the 3.7 backport fixes the
> problem.
If you are fortunate enough to have access to an AIX system, I guess
you have to find out why POWER6 AIX 3.8 and PPC64 AIX 3.8 apparently
work on
Stefan Krah added the comment:
> This code has introduced a regression in AIX in Python 3.7.7
Also this is a rather bold statement since probably no one has ever
run _decimal on AIX with MAX_PREC.
--
___
Python tracker
<https://bugs.pyth
Stefan Krah added the comment:
> This looks like a "new feature/improvement". Why was this code backported to
> a stable version?
Thanks for the lecture. This is an esoteric case between bugfix and
feature that only occurs with very large context precisions.
If Bloom
Change by Stefan Krah :
--
pull_requests: +18317
pull_request: https://github.com/python/cpython/pull/18969
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
memoryview only supports the native format, so I've disabled the
(wrong) test that casts arrays with arbitrary values to _Bool. So
memoryview is done.
IMO the problem in _struct is that it swaps the x->unpack function
for the native one, which does not s
Stefan Krah added the comment:
New changeset f8ce3e2dae277baa2ef92e8a3e935953dc6c3f39 by Miss Islington (bot)
in branch '3.8':
bpo-39689: Do not test undefined casts to _Bool (GH-18964) (#18966)
https://github.com/python/cpython/commit/f8ce3e2dae277baa2ef92e8a3e935953dc6c3f39
Stefan Krah added the comment:
New changeset 636eecc16229432ec8e26e6da287c52f3ca3 by Miss Islington (bot)
in branch '3.7':
bpo-39689: Do not test undefined casts to _Bool (GH-18964) (#18965)
https://github.com/python/cpython/commit/636eecc16229432ec8e26e6da287c52f3ca3
Stefan Krah added the comment:
New changeset 1ae9cde4b2323235b5f9ff4bc76e4175a2257172 by Stefan Krah in branch
'master':
bpo-39689: Do not test undefined casts to _Bool (GH-18964)
https://github.com/python/cpython/commit/1ae9cde4b2323235b5f9ff4bc76e4175a2257172
Change by Stefan Krah :
--
pull_requests: +18312
pull_request: https://github.com/python/cpython/pull/18964
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
The memcpy() is NOT a hack and performs exactly the same operation
as casting the pointer and then dereferencing, only in a manner that
avoids unaligned accesses.
On platforms like x86 the memcpy() is optimized to a simple assignment.
Casting the pointer
Stefan Krah added the comment:
The docs for native mode (we now always assume C99):
"The '?' conversion code corresponds to the _Bool type defined by C99."
The memoryview tests that fail are essentially auto-generated and not
prescriptive. They just happened to work without UBSa
Stefan Krah added the comment:
I checked that NumPy also packs correctly:
>>> import numpy as np
>>> x = np.array([0,1,2,3], dtype=np.bool)
>>> x.tobytes()
b'\x00\x01\x01\x01'
So I vote for not handling incorrectly packed values and removing
"and any
Stefan Krah added the comment:
Okay, in memoryview the cast tests can trigger UB checks. memoryview assumes
that bool is packed correctly, so just casting does not work.
Casting anything to bool is of course a bit silly anyway.
diff --git a/Lib/test/test_buffer.py b/Lib/test/test_buffer.py
Stefan Krah added the comment:
So which memoryview test (unless it is using the struct module)
relies on UB?
--
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
Concerning memoryview, I've looked at this very briefly: memoryview
does not pack values directly, it casts first, which is legal:
#define PACK_SINGLE(ptr, src, type) \
do { \
type x
Stefan Krah added the comment:
Thanks Ned, closing then.
Evgeny, please reopen if you see it again (I ran the tests for about
20 min, way above the usual reproduction time of 1 min).
Thanks for the very instructive test case!
--
status: open -> clo
Change by Stefan Krah :
--
resolution: -> fixed
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Unsubscrib
Change by Stefan Krah :
--
stage: patch review -> resolved
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Un
Stefan Krah added the comment:
#39776 is fixed, but _pydecimal was also affected, see msg363266.
--
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
Setting to release blocker, but please move to deferred again
if a release is almost finished.
--
nosy: +lukasz.langa, ned.deily
priority: deferred blocker -> release blocker
___
Python tracker
<
Stefan Krah added the comment:
Also _pydecimal was affected. This is a modified version of Evgeny's
test case:
from _pydecimal import *
from time import sleep
from random import randint
import sys
sys.setswitchinterval(0.001)
def usleep(x):
sleep(x/100.0)
class Test:
def
Change by Stefan Krah :
Added file: https://bugs.python.org/file48945/pydecimal_cases.zip
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bug
Stefan Krah added the comment:
Since many duplicate tstate ids are generated in the test case
before the crash happens, I guess a set of tstates with the same
id silently uses the cached context of the "winner" tstate.
This can lead to incorrect results without noticing.
-
Change by Stefan Krah :
--
assignee: -> skrah
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Unsubscrib
Change by Stefan Krah :
--
title: Crash in decimal module in heavy-multithreaded scenario ->
PyContextVar_Get(): crash due to race condition in updating tstate->id
___
Python tracker
<https://bugs.python.org/i
Stefan Krah added the comment:
New changeset 852aee69f49c654a03ad1f64d90a78ba8848e2c6 by Stefan Krah in branch
'3.7':
bpo-39776: Lock ++interp->tstate_next_unique_id (GH-18746)
https://github.com/python/cpython/commit/852aee69f49c654a03ad1f64d90a78ba8848e
Stefan Krah added the comment:
New changeset 5a92f42d8723ee865be80f028d402204649da15d by Stefan Krah in branch
'3.8':
bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746) (#18752)
https://github.com/python/cpython/commit/5a92f42d8723ee865be80f028d402204649da
Change by Stefan Krah :
--
pull_requests: +18108
pull_request: https://github.com/python/cpython/pull/18753
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
pull_requests: +18107
pull_request: https://github.com/python/cpython/pull/18752
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset b3b9ade4a3d3fe00d933bcd8fc5c5c755d1024f9 by Stefan Krah in branch
'master':
bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746)
https://github.com/python/cpython/commit/b3b9ade4a3d3fe00d933bcd8fc5c5c755d102
Stefan Krah added the comment:
I think the PR fixes the issue but I have to run longer tests still.
Threads created by PyGILState_Ensure() could have a duplicate tstate->id,
which confused the ContextVar caching machinery.
--
___
Python trac
Change by Stefan Krah :
--
keywords: +patch
pull_requests: +18101
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/18746
___
Python tracker
<https://bugs.python.org/issu
Stefan Krah added the comment:
To make things more clear (it is good that Gregory asked):
ONLY use this flag if you don't use coroutines at all
OR control ALL async libraries in your application.
Random async libraries from PyPI may rely on the
ContextVar and silently give incorrect results
Stefan Krah added the comment:
> If someone builds an interpreter with this configure flag, does it break
> compatibility with anything that code may have started to expect as of 3.7?
Yes, anything that relies on the implicit context being coroutine-safe
does not work. The onl
Stefan Krah added the comment:
For the people who find this issue via a search engine:
1) The defaults in 3.7, 3.8 and 3.9 are unchanged (you get
the ContextVar).
2) ./configure --without-decimal-contextvar enables the exact
TLS context code from 3.3-3.6. There is no new code
Change by Stefan Krah :
--
nosy: +boytsovea
___
Python tracker
<https://bugs.python.org/issue39794>
___
___
Python-bugs-list mailing list
Unsubscribe:
Stefan Krah added the comment:
New changeset c4ca1f8f24118dc5c29e16118fb35a13963af290 by Stefan Krah in branch
'3.7':
[3.7] bpo-39794: Add --without-decimal-contextvar (GH-18702)
https://github.com/python/cpython/commit/c4ca1f8f24118dc5c29e16118fb35a13963af290
Stefan Krah added the comment:
New changeset 4d7012410cf4f91cbca4c406f4747289c2802333 by Stefan Krah in branch
'3.8':
[3.8] bpo-39794: Add --without-decimal-contextvar (GH-18702)
https://github.com/python/cpython/commit/4d7012410cf4f91cbca4c406f4747289c2802333
Change by Stefan Krah :
--
pull_requests: +18074
pull_request: https://github.com/python/cpython/pull/18714
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
pull_requests: +18072
pull_request: https://github.com/python/cpython/pull/18713
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset 815280eb160af637e1347213659f9236adf78f80 by Stefan Krah in branch
'master':
bpo-39794: Add --without-decimal-contextvar (#18702)
https://github.com/python/cpython/commit/815280eb160af637e1347213659f9236adf78f80
Change by Stefan Krah :
--
keywords: +patch
pull_requests: +18064
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/18702
___
Python tracker
<https://bugs.python.org/issu
Stefan Krah added the comment:
Also, when I'm debugging things like #39776, I don't want to
switch between Python versions.
--
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
assignee: -> skrah
components: +Extension Modules
stage: -> needs patch
type: -> behavior
versions: +Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.or
New submission from Stefan Krah :
#39776 has shown that it is hard to understand the interaction between
ContextVars and threading in embedded scenarios.
I want to understand the code again, so I'm adding back a compile time
option to enable the thread local context that was present prior
Stefan Krah added the comment:
> With python 3.7.3 without https://github.com/python/cpython/pull/5278 works
> just fine.
Thanks, I'm now getting the same results as you. Looking at the smaller
test case, I also agree that it should work (as it did in 3.6).
--
ke
Stefan Krah added the comment:
Regarding *my* issue, it could be anything, e.g. a missing call to
PyEval_InitThreads() in 3.6:
"Changed in version 3.7: This function is now called by Py_Initialize(), so you
don’t have to call it yourself anymore."
This is why we need to eliminat
Change by Stefan Krah :
--
resolution: not a bug ->
stage: resolved ->
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list
Stefan Krah added the comment:
Note that my pybind11 is from GitHub master, it can also be a pybind11
issue.
It is interesting that you cannot reproduce your original issue with
3.6, so I'm reopening this issue.
I think we need a reproducer without pybind11 though, could you
tweak Programs
Stefan Krah added the comment:
This is 3.6.7, compiled --with-pydebug:
$ ./main
Aborted (core dumped)
(gdb) bt
#0 0x7f9974077428 in __GI_raise (sig=sig@entry=6) at
../sysdeps/unix/sysv/linux/raise.c:54
#1 0x7f997407902a in __GI_abort () at abort.c:89
#2 0x0056e2d1
Stefan Krah added the comment:
I built your example with 3.6:
git clone https://github.com/pybind/pybind11
wget https://bugs.python.org/file48923/decimal_crash.zip
unzip decimal_crash.zip
git checkout v3.6.7
./configure --with-pydebug
make
g++ -std=c++11 -pthread -Wno-unused-result -Wsign
Change by Stefan Krah :
--
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Stefan Krah added the comment:
I've briefly looked at the zip archive. Without going much into
the C++ module as a whole, this should not be done:
gil_unlocker.UnlockGILAndSleep()
self.val = decimal.Decimal(1) / decimal.Decimal(7)
gil_unlocker.UnlockGILAndSleep()
If you want C
Stefan Krah added the comment:
Before I look at the example code: Can you also reproduce this with
Python 3.6? The threading code in _decimal was changed to a ContextVar
in 3.7.
There's a high chance though that the problem is in the c++ module.
--
nosy: +skrah
Stefan Krah added the comment:
For me even a mail with a single line would be too much. I can filter that in
my mail client but not on GitHub.
Speaking about that, I also don't want to get mail from Bevedere stating that
I, in fact, have signed a CLA any time I open a PR
Stefan Krah added the comment:
New changeset c6ecd9c14081a787959e13df33e250102a658154 by Miss Islington (bot)
in branch '3.8':
bpo-39576: Clarify the word size for the 32-bit build. (GH-18616) (#18618)
https://github.com/python/cpython/commit/c6ecd9c14081a787959e13df33e250102a658154
Stefan Krah added the comment:
New changeset 24c570bbb82a7cb70576c253a73390accfa7ed78 by Miss Islington (bot)
in branch '3.7':
bpo-39576: Clarify the word size for the 32-bit build. (GH-18616) (#18617)
https://github.com/python/cpython/commit/24c570bbb82a7cb70576c253a73390accfa7ed78
Stefan Krah added the comment:
New changeset b76518d43fb82ed9e5d27025d18c90a23d525c90 by Stefan Krah in branch
'master':
bpo-39576: Clarify the word size for the 32-bit build. (#18616)
https://github.com/python/cpython/commit/b76518d43fb82ed9e5d27025d18c90a23d525c90
Change by Stefan Krah :
--
pull_requests: +17981
pull_request: https://github.com/python/cpython/pull/18616
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
stage: patch review -> resolved
___
Python tracker
<https://bugs.python.org/issue39576>
___
___
Python-bugs-list mailing list
Un
Stefan Krah added the comment:
libmpdec and the docs are done, the question remains what to do with
decimal.py.
It has the same behavior, but I don't think users are running
decimal.py with very large precisions.
Anyway, unassigning myself in case anyone else wants to work on a patch
Stefan Krah added the comment:
New changeset d6965ff026f35498e554bc964ef2be8f4d80eb7f by Miss Islington (bot)
in branch '3.8':
bpo-39576: docs: set context for decimal arbitrary precision arithmetic
(GH-18594) (#18597)
https://github.com/python/cpython/commit
Stefan Krah added the comment:
New changeset 00e45877e33d32bb61aa13a2033e3bba370bda4d by Miss Islington (bot)
in branch '3.7':
bpo-39576: docs: set context for decimal arbitrary precision arithmetic
(GH-18594) (#18596)
https://github.com/python/cpython/commit
Stefan Krah added the comment:
New changeset a025d4ca99fb4c652465368e0b4eb03cf4b316b9 by Stefan Krah in branch
'master':
bpo-39576: docs: set context for decimal arbitrary precision arithmetic (#18594)
https://github.com/python/cpython/commit/a025d4ca99fb4c652465368e0b4eb03cf4b316b9
Stefan Krah added the comment:
They are allowed failures but the build is still marked in red:
https://github.com/python/cpython/pull/18567
So if you look at the front page you have to click through red results
only to find that the reason is code coverage
Stefan Krah added the comment:
Updated docs:
https://github.com/python/cpython/pull/18594
The PR uses some of Tim's suggestions while also explaining how to
calculate the amount of memory used in a single large decimal.
Hopefully it isn't too much information
Change by Stefan Krah :
--
pull_requests: +17961
pull_request: https://github.com/python/cpython/pull/18594
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
"So non-integer powers are left out" in isolation would indeed be
wrong, but actual sentence is unambiguously qualified with:
"... since _decimal has no notion of exact non-integer powers yet.",
which clearly states that exact non-
Stefan Krah added the comment:
Vedran, msg362365 is meant to say:
"This PR implements $SOMEWHAT_MATHEMATICAL_SPEC except for inexact power."
Had I put the caveat inside the statement as well, the message would have been:
"Thi
Stefan Krah added the comment:
I'd definitely disable the automatic comment and prefer that the build
happens on buildbot.python.org rather than affecting the GitHub build
status.
--
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset b6271025c640c228505dc9f194362a0c2ab81c61 by Miss Islington (bot)
in branch '3.8':
bpo-39576: Prevent memory error for overly optimistic precisions (GH-18581)
(#18584)
https://github.com/python/cpython/commit
Stefan Krah added the comment:
New changeset c6f95543b4832c3f0170179da39bcf99b40a7aa8 by Miss Islington (bot)
in branch '3.7':
bpo-39576: Prevent memory error for overly optimistic precisions (GH-18581)
(#18585)
https://github.com/python/cpython/commit
Stefan Krah added the comment:
New changeset 90930e65455f60216f09d175586139242dbba260 by Stefan Krah in branch
'master':
bpo-39576: Prevent memory error for overly optimistic precisions (GH-18581)
https://github.com/python/cpython/commit/90930e65455f60216f09d175586139242dbba260
New submission from Stefan Krah :
The automated code coverage on GitHub is quite inaccurate and needlessly flags
PRs as red.
I'd prefer to make this opt-in.
--
messages: 362367
nosy: skrah
priority: normal
severity: normal
status: open
title: Disable code coverage
Stefan Krah added the comment:
BTW, this PR implements the invariant:
"If there exists an exact result at a lower precision, this
result should also be returned at MAX_PREC (without MemoryError)".
So non-integer powers are left out, since _decimal has no notion
of exact non-inte
Stefan Krah added the comment:
Fortunately libmpdec raises MemoryError almost instantaneously, so the PR
retries
the affected operations with estimated upper bounds for exact results without
slowing down the common case.
The docs still need updating because people will still wonder why 1
Change by Stefan Krah :
--
keywords: +patch
pull_requests: +17951
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/18581
___
Python tracker
<https://bugs.python.org/issu
Change by Stefan Krah :
--
status: closed -> open
___
Python tracker
<https://bugs.python.org/issue39701>
___
___
Python-bugs-list mailing list
Unsubscrib
Stefan Krah added the comment:
Re-opening this issue, Azure fails too often:
https://github.com/python/cpython/pull/18577
https://dev.azure.com/Python/cpython/_build/results?buildId=58220=logs=c83831cd-3752-5cc7-2f01-8276919eb334
--
___
Python
Stefan Krah added the comment:
Closing and reopening the PR helped.
--
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/i
New submission from Stefan Krah :
There is no status report, no details link and manually committing is
prohibited:
https://github.com/python/cpython/pull/18569
--
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
nosy: skrah
priority: normal
severity: normal
status: open
title: Azure Pipelines PR broken
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
The change looks reasonable, but unfortunately this is a long-standing
behavior that originates from before the Python-3.3 memoryview rewrite.
It is also present in 2.7 (the previous implementation) and documented
in 3.3:
https://docs.python.org/3/library
New submission from Stefan Krah :
As mentioned in the PR, I don't see sufficient evidence that backticks are
legacy. So I'll close this.
--
assignee: -> skrah
nosy: +skrah
resolution: -> rejected
stage: patch review -> resolved
status: open
Stefan Krah added the comment:
> This isn't purely academic. The `decimal` docs, at the end:
Yes, that is a good point. I think for libmpdec I'll just do a trial divmod if
prec > BIGNUM_THRESHOLD.
> Perhaps the only thing to be done is to add words to the part of
Stefan Krah added the comment:
The feature would be nice to have; however, if you choose the precision to
match the amount of available RAM things work (I have 8GB here, one word in the
coefficient has 19 digits for the 4 bit version):
>>> from decimal import *
>>&
Stefan Krah added the comment:
MAX_PREC is chosen so that 5*MAX_PREC does not overflow 32-bit or 64-bit signed
integers. This eliminates many overflow checks for the exponent.
Updating the exponent is (perhaps surprisingly) quite performance sensitive,
that's why the 32-bit build does
Stefan Krah added the comment:
With _pydecimal the memory also grows very slowly (I didn't have the patience
to wait for MemoryError).
I'm pretty sure decNumber also does the same, it's just easier to implement
and does not slow down division for small numbers.
libmpdec always favors
Stefan Krah added the comment:
Why would one not abort() in release mode? If the cast is inexact, the results
will usually be so wrong that even on a web server a hard exit is preferable.
I don't think the check costs much time with branch prediction.
--
nosy: +skrah
Stefan Krah added the comment:
Is it available with -std=c11? It is a bit strange that we use -std=c99. I
thought that header is C11:
https://en.cppreference.com/w/c/atomic
--
nosy: +skrah
___
Python tracker
<https://bugs.python.org/issue37
Stefan Krah added the comment:
> Or do you care about *using* API with underscore? If so, I'm OK to stop
> changing some callers which are not tightly coupled with Python.
I care about this one. Indeed I think underscore functions should be used in
strategic places inside th
Stefan Krah added the comment:
It adds yet another special case underscore function that one cannot use in
external projects. So I would not say that is simpler.
Has there been any performance measurement at all?
--
___
Python tracker
<ht
201 - 300 of 3373 matches
Mail list logo
