[issue44243] tkinter button colors on Mac

2021-05-27 Thread Mark Roseman


Mark Roseman  added the comment:

Let it go. Changing the Python docs to a behaviour that isn't guaranteed by the 
underlying library is a virtual guarantee that a later version of Tk (or even 
the way the API it uses behaves on another version of macOS) will have some 
other (unrelated) modifications that change the current behaviour to something 
else.

--

___
Python tracker 
<https://bugs.python.org/issue44243>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44250] Link to ibera IRC goes to 404 page

2021-05-27 Thread Mark Winterbottom


New submission from Mark Winterbottom :

The link to "libera" on https://www.python.org/community/irc/ goes to 
http://libera.chat/t which is 404 not found.

--
messages: 394561
nosy: londonappdev
priority: normal
severity: normal
status: open
title: Link to ibera IRC goes to 404 page

___
Python tracker 
<https://bugs.python.org/issue44250>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-26 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 6cc800d3634fdd002b986c3ffe6a3d5540f311a0 by Eric Snow in branch 
'main':
bpo-43693: Clean up the PyCodeObject fields. (GH-26364)
https://github.com/python/cpython/commit/6cc800d3634fdd002b986c3ffe6a3d5540f311a0


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44240] Incorrect behavior of LOAD_ATTR due to overflow in tp_version

2021-05-26 Thread Mark Shannon


Mark Shannon  added the comment:

It is extremely unlikely, I agree. But not impossible.

I plan to fix it for 3.11. Once I've done that we can decide if backports are 
worth it.

--

___
Python tracker 
<https://bugs.python.org/issue44240>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44240] Incorrect behavior of LOAD_ATTR due to overflow in tp_version

2021-05-26 Thread Mark Shannon


New submission from Mark Shannon :

Should the tp_version overflow, and wrap around to a value previously used in 
the opcache for LOAD_ATTR, then LOAD_ATTR could produce the wrong value.

It will take a fair bit of contrivance and a long time to do this, but it is 
possible:
Run some code ~2000 times to get the version cached.
Change an attibute of the type about 4 billion times.
Rerun the original code.



Invalidating all the opcaches is going to be a huge pain, so I propose not 
allowing the version to overflow but partitioning the 32 bit space something 
like this:

Top 20 bits: Unique per-class ID, 0 and 0xF are reserved.
Low 12 bits: Per-class version.

tp_version == 0 that no version has been assigned to this class, as yet.
(tp_version & 0xFFF) == 0 means that the version tag is temporarily invalid
tp_version == 0x means that the version tag is permanently invalid

If (tp_version & 0xFFF) != 0 and tp_version != 0x, then the combined 32 
bits represents a unique tag of the class's state as it does now.

Should the low bits of a class hit 0xFFF then all 32 bits are set to 
0x, and we can't cache its version any more.
If a class has been changed a 1000 times, then there is unlikely to be much 
value in caching it anyway.

--
components: Interpreter Core
messages: 394442
nosy: Mark.Shannon, pablogsal
priority: normal
severity: normal
stage: needs patch
status: open
title: Incorrect behavior of LOAD_ATTR  due to overflow in tp_version
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue44240>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37439] Add random.binomialvariate()

2021-05-25 Thread Mark Dickinson


Mark Dickinson  added the comment:

Nope, that's the wrong paper. It looks as though this is the right one, but 
it's hidden behind a paywall: https://dl.acm.org/doi/abs/10.1145/42372.42381

--

___
Python tracker 
<https://bugs.python.org/issue37439>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37439] Add random.binomialvariate()

2021-05-25 Thread Mark Dickinson


Mark Dickinson  added the comment:

I think the NumPy implementation may be from here: 
https://core.ac.uk/download/pdf/11007254.pdf

(though I'm struggling to find a clear citation in the NumPy source)

--

___
Python tracker 
<https://bugs.python.org/issue37439>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44206] Add a version number to dict keys.

2021-05-25 Thread Mark Shannon


Mark Shannon  added the comment:

Which optimizations?

LOAD_GLOBAL:
Using a keys version instead of a whole dict version means that LOAD_GLOBAL 
won't leak references. It also means that we can (in the future) remove the PEP 
509 version and save 8 bytes per dict.

LOAD_ATTR:
_PyDict_GetItemHint() still has to do quite a lot of work compared to a version 
check.
The hint approach can't quickly tell us whether a name is not in a dictionary, 
which is needed for optimizing non-descriptor class attributes.

LOAD_METHOD:
Because functions are non-overriding descriptors we need to quickly check that 
the instance does not have an attribute shadowing the method.


Why is 32 bits enough?

Because the version is reset to zero, whenever the dict keys changes, and only 
set to non-zero when we explicitly ask for it when optimizing. 4 billion 
optimization events is a lot.
It can't overflow, it just becomes useless when we reach UINT_MAX.
Using 64 bits would just waste memory.


Overall, versioning the dictionary's keys is more useful and more compact than 
versioning the dictionary as a whole.

--

___
Python tracker 
<https://bugs.python.org/issue44206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33213] crypt function not hashing properly on Mac (uses a specific salt)

2021-05-25 Thread Mark Dickinson


Change by Mark Dickinson :


--
nosy:  -mark.dickinson

___
Python tracker 
<https://bugs.python.org/issue33213>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44206] Add a version number to dict keys.

2021-05-24 Thread Mark Shannon


Mark Shannon  added the comment:

http://theses.gla.ac.uk/2975/1/2011shannonphd.pdf page 128.

It means we don't need to cache a pointer to the keys, just the version number.
The version number is half the size (for 64 bit machines) and using it means 
that we don't leak keys.

--

___
Python tracker 
<https://bugs.python.org/issue44206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44206] Add a version number to dict keys.

2021-05-24 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24925
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/26333

___
Python tracker 
<https://bugs.python.org/issue44206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44206] Add a version number to dict keys.

2021-05-24 Thread Mark Shannon


Mark Shannon  added the comment:

The version is on the dict keys. It changes if the dict keys object changes in 
a meaningful way.

So, no it doesn't change if a value is changed.
Nor would it change if the keys were shared and a new key-value pair is added 
to a dict, but that key was already in the shared keys.

--

___
Python tracker 
<https://bugs.python.org/issue44206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44032] Function locals and evaluation stack should be stored in a contiguous, per-thread stack

2021-05-21 Thread Mark Shannon


Mark Shannon  added the comment:

What's the test case, exactly?

ref.py for the other issue doesn't crash if I change "func.py" to "ref.py"
otherwise it just complains that "func.py" doesn't exist.

--

___
Python tracker 
<https://bugs.python.org/issue44032>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44032] Function locals and evaluation stack should be stored in a contiguous, per-thread stack

2021-05-21 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +24891
pull_request: https://github.com/python/cpython/pull/26285

___
Python tracker 
<https://bugs.python.org/issue44032>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44207] Add a version number to Python functions

2021-05-21 Thread Mark Shannon


New submission from Mark Shannon :

In order to specialize calls to Python functions, or to inline them, we need to 
know that the code object has not changed. It is also useful to know that the 
globals, builtins and various defaults haven't changed either. Rather than 
attempting to check these individually it is much simpler and faster to check a 
version number.

--
assignee: Mark.Shannon
messages: 394124
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Add a version number to Python functions
type: performance

___
Python tracker 
<https://bugs.python.org/issue44207>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44206] Add a version number to dict keys.

2021-05-21 Thread Mark Shannon


Mark Shannon  added the comment:

The memory saving comes from converting:

Py_ssize_t dk_size;
dict_lookup_func dk_lookup;

to:

   uint8_t dk_log2_size;
   uint8_t dk_loopup_kind; /* Only 3 possible values */
   uint32_t dk_version;

which saves 8 bytes on a 64 bit machine (no saving on a 32 bit machine).

--

___
Python tracker 
<https://bugs.python.org/issue44206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44206] Add a version number to dict keys.

2021-05-21 Thread Mark Shannon


New submission from Mark Shannon :

Add a version number to dict keys.

PEP 509 added a version number to dicts. Unfortunately this is no use for 
optimizing attribute loads and store on instances.
We need to know whether the keys are as expected, not the dict as that is 
likely to be different each time.

We can add a 32 bit version number and actually reduce memory use by taking 
advantage of the redundancy in the rest of the keys object.

--
assignee: Mark.Shannon
messages: 394120
nosy: Mark.Shannon, methane, vstinner
priority: normal
severity: normal
stage: needs patch
status: open
title: Add a version number to dict keys.
type: performance

___
Python tracker 
<https://bugs.python.org/issue44206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44187] Implement infrastructure for quickening and specializing

2021-05-20 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24868
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26264

___
Python tracker 
<https://bugs.python.org/issue44187>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44187] Implement infrastructure for quickening and specializing

2021-05-20 Thread Mark Shannon


New submission from Mark Shannon :

As described in PEP 659 (Specializing Adaptive Interpreter) the first part of 
implementing specialization is to implement the machinery to do the quickening.

Conceptually, this is fairly simple: add a new field to the code object, which 
points to the first instruction in the bytecode.

When quickening, we create a new array, copy the old bytecode into it, and make 
the new field point to it.

Without any specialization or superinstructions, this will just waste memory.
However, it will pay off soon enough as we implement superinstructions, remove 
the old "opcache" and add new specializations.

We expect to see worthwhile speed ups with just superinstructions, and large 
speedups when all the above features have been implemented.

--
assignee: Mark.Shannon
messages: 394013
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Implement infrastructure for quickening and specializing
type: performance

___
Python tracker 
<https://bugs.python.org/issue44187>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28937] str.split(): allow removing empty strings (when sep is not None)

2021-05-20 Thread Mark Bell


Mark Bell  added the comment:

Thank you very much for confirming these test cases. Using these I believe that 
I have now been able to complete a patch that would implement this feature. The 
PR is available at https://github.com/python/cpython/pull/26222. As I am a 
first-time contributor, please could a maintainer approve running the CI 
workflows so that I can confirm that all the (new) tests pass.

--

___
Python tracker 
<https://bugs.python.org/issue28937>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28937] str.split(): allow removing empty strings (when sep is not None)

2021-05-18 Thread Mark Bell


Change by Mark Bell :


--
pull_requests: +24839
pull_request: https://github.com/python/cpython/pull/26222

___
Python tracker 
<https://bugs.python.org/issue28937>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28937] str.split(): allow removing empty strings (when sep is not None)

2021-05-18 Thread Mark Bell


Mark Bell  added the comment:

So I think I agree with you about the difference between .split() and .split(' 
'). However wouldn't that mean that
'   x y z'.split(maxsplit=1, keepempty=False) == ['x', 'y z']

since it should do one split.

--

___
Python tracker 
<https://bugs.python.org/issue28937>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28937] str.split(): allow removing empty strings (when sep is not None)

2021-05-18 Thread Mark Bell


Mark Bell  added the comment:

> suggests that empty strings don't count towards maxsplit

Thank you for the confirmation. Although just to clarify I guess you really 
mean "empty strings *that are dropped from the output* don't count towards 
maxsplit". Just to double check this, what do we expect the output of

'   x y z'.split(maxsplit=1, keepempty=True)

to be?

I think it should be ['', '  x y z'] since in this case we are retaining empty 
strings and they should count towards the maxsplit.

(In the current patch this crashes with a core dump since it tries to write to 
unallocated memory)

--

___
Python tracker 
<https://bugs.python.org/issue28937>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28937] str.split(): allow removing empty strings (when sep is not None)

2021-05-18 Thread Mark Bell

Mark Bell  added the comment:

So I have taken a look at the original patch that was provided and I have been 
able to update it so that it is compatible with the current release. I have 
also flipped the logic in the wrapping functions so that they take a 
`keepempty` flag (which is the opposite of the `prune` flag). 

I had to make a few extra changes since there are now some extra catches in 
things like PyUnicode_Split which spot that if len(self) > len(sep) then they 
can just return [self]. However that now needs an extra test since that 
shortcut can only be used if len(self) > 0. You can find the code here: 
https://github.com/markcbell/cpython/tree/split-keepempty

However in exploring this, I'm not sure that this patch interacts correctly 
with maxsplit. For example, 
'   x y z'.split(maxsplit=1, keepempty=True)
results in
['', '', 'x', 'y z']
since the first two empty strings items are "free" and don't count towards the 
maxsplit. I think the length of the result returned must be <= maxsplit + 1, is 
this right?

I'm about to rework the logic to avoid this, but before I go too far could 
someone double check my test cases to make sure that I have the correct idea 
about how this is supposed to work please. Only the 8 lines marked "New case" 
show new behaviour, all the other come from how string.split works currently. 
Of course the same patterns should apply to bytestrings and bytearrays.

''.split() == []
''.split(' ') == ['']
''.split(' ', keepempty=False) == []# New case

'  '.split(' ') == ['', '', '']
'  '.split(' ', maxsplit=1) == ['', ' ']
'  '.split(' ', maxsplit=1, keepempty=False) == [' ']# New case

'  a b c  '.split() == ['a', 'b', 'c']
​'  a b c  '.split(maxsplit=0) == ['a b c  ']
​'  a b c  '.split(maxsplit=1) == ['a', 'b c  ']

'  a b c  '.split(' ') == ['', '', 'a', 'b', 'c', '', '']
​'  a b c  '.split(' ', maxsplit=0) == ['  a b c  ']
​'  a b c  '.split(' ', maxsplit=1) == ['', ' a b c  ']
​'  a b c  '.split(' ', maxsplit=2) == ['', '', 'a b c  ']
​'  a b c  '.split(' ', maxsplit=3) == ['', '', 'a', 'b c  ']
​'  a b c  '.split(' ', maxsplit=4) == ['', '', 'a', 'b', 'c  ']
​'  a b c  '.split(' ', maxsplit=5) == ['', '', 'a', 'b', 'c', ' ']
​'  a b c  '.split(' ', maxsplit=6) == ['', '', 'a', 'b', 'c', '', '']

​'  a b c  '.split(' ', keepempty=False) == ['a', 'b', 'c']# New case
​'  a b c  '.split(' ', maxsplit=0, keepempty=False) == ['  a b c  ']# 
New case
​'  a b c  '.split(' ', maxsplit=1, keepempty=False) == ['a', 'b c  ']# 
New case
​'  a b c  '.split(' ', maxsplit=2, keepempty=False) == ['a', 'b', 'c  ']   
 # New case
​'  a b c  '.split(' ', maxsplit=3, keepempty=False) == ['a', 'b', 'c', ' 
']# New case
​'  a b c  '.split(' ', maxsplit=4, keepempty=False) == ['a', 'b', 'c']
# New case

--
nosy: +Mark.Bell

___
Python tracker 
<https://bugs.python.org/issue28937>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39394] re: DeprecationWarning for `flag not at the start of expression` is cutoff too early

2021-05-17 Thread Mark Shannon


Mark Shannon  added the comment:

I have to admit that I find the truncated version more readable.

Some sort of truncation is useful, as a regex could be thousands of character 
long.

Adding the offset to the warning message seems like a useful addition.

--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue39394>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43504] Site linked in docs, effbot.org, down

2021-05-13 Thread Mark Roseman


Mark Roseman  added the comment:

I'd argue for removing the links altogether, given the material is very 
outdated and from what I recall anything that was there is better covered now 
by TkDocs, Shipman, or other resources.

--

___
Python tracker 
<https://bugs.python.org/issue43504>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-13 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 0acdf255a51b836c0b44f3676797620322974af3 by Mark Shannon in 
branch '3.10':
[3.10] bpo-43933: Force RETURN_VALUE bytecodes to have line numbers (GH-26061)
https://github.com/python/cpython/commit/0acdf255a51b836c0b44f3676797620322974af3


--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44115] Improve conversions for fractions

2021-05-13 Thread Mark Dickinson


Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 
<https://bugs.python.org/issue44115>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44032] Function locals and evaluation stack should be stored in a contiguous, per-thread stack

2021-05-12 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24716
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26076

___
Python tracker 
<https://bugs.python.org/issue44032>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-12 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +24702
pull_request: https://github.com/python/cpython/pull/26061

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-05-12 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +24699
pull_request: https://github.com/python/cpython/pull/26059

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-12 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +24694
pull_request: https://github.com/python/cpython/pull/26054

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44088] traced line number can be None

2021-05-12 Thread Mark Shannon


Mark Shannon  added the comment:

Duplicate of https://bugs.python.org/issue43933

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue44088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-05-11 Thread Mark Shannon


Mark Shannon  added the comment:

It is very little effort to add back the old function, so that isn't the 
problem. It won't work properly, but it never did anyway. So I guess that's 
sort of compatible.

Maybe the best thing is to put a big red warning in the docs and hope that 
warns away people from using it?

--

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-05-11 Thread Mark Shannon


Mark Shannon  added the comment:

I know PyCode_NewWithPosOnlyArgs is declared as "PyAPI_FUNC" but that can't 
make it part of the ABI unless it has stable behavior.
It can't have stable behavior because its inputs are complex, undefined, have 
altered semantics and are interlinked in complex ways.

Passing the same arguments to PyCode_NewWithPosOnlyArgs for both 3.9 and 3.10 
will cause one or other version to crash (interpreter crash, not just program 
crash).


We need to stop adding "PyAPI_FUNC" to everything.
Adding a PyAPI_FUNC does not magically make for ABI compatibility, there is a 
lot more to it than that.

The only sane ways to construct a code object are to load it from disk, to 
compile an AST, or to use
codeobject.replace(). Any purported ABI compatibility claims are just 
misleading and a trap.


I can revert the API changes and add a new function, but I think that is 
dangerously misleading. A compilation error is preferable to an interpreter 
crash.

--

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-05-10 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +24671
pull_request: https://github.com/python/cpython/pull/26021

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be.

2021-05-10 Thread Mark Shannon


Mark Shannon  added the comment:

But what does "use it" mean?
What does setting `tstate->use_tracing = 1` do?
There is no documented behavior, so how do we know what assumptions people are 
making about what happens when they set some field to 1?

As I said, we could keep the field and ignore it, but that seems worse.

--

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be.

2021-05-10 Thread Mark Shannon


Mark Shannon  added the comment:

If there is no C-API function that supports your needs, feel free to suggest 
one.

--

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-10 Thread Mark Shannon


Change by Mark Shannon :


--
Removed message: https://bugs.python.org/msg393388

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-10 Thread Mark Shannon


Mark Shannon  added the comment:

If there is no C-API function that supports your needs, feel free to suggest 
one.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43760] The DISPATCH() macro is not as efficient as it could be.

2021-05-10 Thread Mark Shannon


Mark Shannon  added the comment:

At yappi/_yappi.c:1261 sets an undocumented field on a CPython internal data 
structure.

What did you believe that was supposed to do? use_tracing is not documented 
anywhere.

We could add the field back and ignore it, but I doubt that would help you much.

--

___
Python tracker 
<https://bugs.python.org/issue43760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-10 Thread Mark Shannon


Mark Shannon  added the comment:

Ned, no line numbers should never be None.
(Except under very unusual circumstances, like the trace function raising an 
error)

Taking a look at the disassembly of execsitecustomize, there is a return with 
no line number, which shouldn't happen.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-05-10 Thread Mark Shannon


Mark Shannon  added the comment:

Thanks everyone for the triaging and fixing.

--

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44080] Bias in random.choices(long_sequence)

2021-05-09 Thread Mark Dickinson


Mark Dickinson  added the comment:

This is similar to #9025 (for randrange), and in principle the random.choices 
case without explicit weights could be fixed similarly. But I'm not sure it's 
worth it, for a couple of reasons:

- The problem only really affects "virtual" population sequences like 
``range``, where the actual population isn't realised in memory. For 
populations in list form, in practice the population size will be limited to a 
few billion, and I think you'd be hard pushed to demonstrate statistically 
detectable bias at that size. (That is, my claim is that it would be 
essentially impossible to distinguish the existing "random.choices" 
implementation from a "fixed" unbiased version based on inputs and outputs 
alone.)

- The speed of random.choices is a particular selling point - we don't have 
anything else in the random module that lets you generate millions of variates 
at once. I'd expect replacing `floor(random() * n)` with `self._randbelow(n)` 
to cause a significant speed regression.

- For explicit weights, there isn't a practical route to handling those that 
doesn't involve use of floating-point in general, so for those we have to 
accept some tiny bias anyway. (Yes, we could technically do something unbiased 
for the case of integer-only weights, but that seems like significant work for 
a special case that I'd expect just doesn't matter that much in practice.) And 
again in this case, the weights list will usually be a "real" list, so we're 
again limited to populations of a few billion and it'll be hard to demonstrate 
statistically detectable bias.

@Dennis: for interests sake, do you have bandwidth to do some timings of the 
current "[population[floor(random() * n)] for i in _repeat(None, k)]" versus 
something like "[population[_randbelow(n)] for i in _repeat(None, k)]" (after 
putting "_randbelow = self._randbelow" to save an attribute lookup on each 
iteration)?

--

___
Python tracker 
<https://bugs.python.org/issue44080>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44080] Bias in random.choices(long_sequence)

2021-05-09 Thread Mark Dickinson


Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 
<https://bugs.python.org/issue44080>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44078] Output relative path when using PurePath.relative_to

2021-05-08 Thread Mark Hammond


New submission from Mark Hammond :

Comparing two paths, PurePath.relative_to fails with ValueError if the second 
path is not in the first.

>>> Path('/a/b').relative_to('/a/b/c')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.9/pathlib.py", line 928, in relative_to
raise ValueError("{!r} is not in the subpath of {!r}"
ValueError: '/a/b' is not in the subpath of '/a/b/c' OR one path is relative 
and the other is absolute.

Please extend PurePath.relative_to so it is able to output a relative path in 
form of '..' instead of an error. Currently, the only way to do this is to use 
relpath from os.path but it would be very useful if Path was able to output the 
relative path.

--
components: Library (Lib)
messages: 393262
nosy: mhammondr
priority: normal
severity: normal
status: open
title: Output relative path when using PurePath.relative_to
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue44078>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44056] Incorrect line number for syntax error.

2021-05-07 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> needs patch
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue44056>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44054] 2**53+1 != float(2**53+1)

2021-05-06 Thread Mark Dickinson


Mark Dickinson  added the comment:

Closing here, since this isn't a bug. But I'd still like to understand better 
what Stefan meant by "That's also what the code says."

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue44054>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44054] 2**53+1 != float(2**53+1)

2021-05-06 Thread Mark Dickinson


Mark Dickinson  added the comment:

> I have a vague memory that that's the way it *did* work once upon a time

Here we go: https://bugs.python.org/issue513866
And the corresponding commit, from September 2004: 
https://github.com/python/cpython/commit/307fa78107c39ffda1eb4ad18201d25650354c4e

--

___
Python tracker 
<https://bugs.python.org/issue44054>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44054] 2**53+1 != float(2**53+1)

2021-05-06 Thread Mark Dickinson


Mark Dickinson  added the comment:

> But the way I would have expected this to work is that a comparison of an 
> integer to a float would first convert the integer to a float

I have a vague memory that that's the way it *did* work once upon a time, many 
many decades ago. But an equality comparison between int and float that simply 
converted the int to a float would break transitivity of equality, leading to 
issues with containment in sets, dicts and lists. Given

n = 2**53
x = 2.**53

if equality comparison worked as you describe you'd have n == x and x == n + 1, 
so to keep transitivity you'd have to make n == n + 1.

In short, the behaviour is very much deliberate.

> That's also what the code says.

Do you have a pointer? This may be a comment bug.

--

___
Python tracker 
<https://bugs.python.org/issue44054>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44056] Incorrect line number for syntax error.

2021-05-06 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24606
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/25939

___
Python tracker 
<https://bugs.python.org/issue44056>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44056] Incorrect line number for syntax error.

2021-05-06 Thread Mark Shannon


New submission from Mark Shannon :

Consider this function, which has a syntax error on line 4.

>>> def f():
... try: 
... 1/0
... except:
... pass
... except Exception: 
... pass

3.9 reports an incorrect line number of 3.
3.10b reports an even more incorrect line number of -1.

Although I've marked this as a "Parser" bug, the offending code is in the 
compiler.

For 3.11, this is fixed by https://github.com/python/cpython/pull/25729

--
assignee: Mark.Shannon
components: Parser
messages: 393083
nosy: Mark.Shannon, lys.nikolaou, pablogsal
priority: release blocker
severity: normal
stage: needs patch
status: open
title: Incorrect line number for syntax error.
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44056>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44032] Function locals and evaluation stack should be stored in a contiguous, per-thread stack

2021-05-04 Thread Mark Shannon


Change by Mark Shannon :


--
title: Store locals and evaluation stack should be stored in a contiguous, 
per-thread stack -> Function locals and evaluation stack should be stored in a 
contiguous, per-thread stack

___
Python tracker 
<https://bugs.python.org/issue44032>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44032] Store locals and evaluation stack should be stored in a contiguous, per-thread stack

2021-05-04 Thread Mark Shannon


New submission from Mark Shannon :

Currently, the local variables (inc. cell and free vars) for functions and the 
evaluation stack are kept in per-activation chunks in the frame object. This is 
not a good design for modern hardware.
The local variables and stack are amongst the hottest memory in the VM and 
should be stored in a contiguous stack in memory.

Allocating a per-thread stack would improve memory locality considerably, and 
pave the way to allocating heap objects for function activations lazily, only 
when needed for debugging and introspection.

Generators would still need heap allocated locals and stack, but that would be 
no worse than currently.

--
assignee: Mark.Shannon
components: Interpreter Core
messages: 392906
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Store locals and evaluation stack should be stored in a contiguous, 
per-thread stack
type: performance

___
Python tracker 
<https://bugs.python.org/issue44032>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38135] Depth first search in compile.c creates wrong BB order for certain CFG.

2021-05-04 Thread Mark Shannon


Mark Shannon  added the comment:

This seems to have been fixed sometime.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38135>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43054] What does the existence of a struct in a header file imply about the C-API

2021-05-04 Thread Mark Shannon


Mark Shannon  added the comment:

Thanks for the responses.
Probably nothing to do for now.

--
resolution:  -> postponed
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43054>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42067] Type annotation in for-loops

2021-05-04 Thread Mark Shannon


Mark Shannon  added the comment:

Definitely one for Python ideas.
Quite a good idea though.

--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue42067>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43977] Implement the latest semantics for PEP 634 for matching collections

2021-05-02 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43977>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43977] Implement the latest semantics for PEP 634 for matching collections

2021-05-02 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 33ec88ac81f23668293d101b83367b086c795e5e by Mark Shannon in 
branch 'master':
bpo-43977: Make sure that tp_flags for pattern matching are inherited 
correctly. (GH-25813)
https://github.com/python/cpython/commit/33ec88ac81f23668293d101b83367b086c795e5e


--

___
Python tracker 
<https://bugs.python.org/issue43977>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43977] Implement the latest semantics for PEP 634 for matching collections

2021-05-02 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +24500
pull_request: https://github.com/python/cpython/pull/25813

___
Python tracker 
<https://bugs.python.org/issue43977>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43996] Doc for mutable sequence pop() method implies argument is a slice or sequence.

2021-04-30 Thread Mark Sapiro


Mark Sapiro  added the comment:

Thank you for the explanation which I understand and accept. I also fully (or 
maybe not quite fully) understand the use of square brackets to indicate 
optional arguments. It's just that in the context of the table at 
https://docs.python.org/3/library/stdtypes.html#mutable-sequence-types every 
other use of square brackets indicates a list or a slice and that's what 
confused me. Granted, all the other square bracket usage was not around a 
method argument, and I accept that the doc is correct, but I still found it 
confusing.

--

___
Python tracker 
<https://bugs.python.org/issue43996>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43996] Doc for mutable sequence pop() method implies argument is a slice or sequence.

2021-04-30 Thread Mark Sapiro


New submission from Mark Sapiro :

In several places in the documentation including:

```
grep -rn 'pop.\[i\]'
Lib/pydoc_data/topics.py:13184: '| "s.pop([i])"   | 
retrieves the item at *i* '
Lib/pydoc_data/topics.py:13647: '| "s.pop([i])" 
  | retrieves the item at '
Doc/tutorial/datastructures.rst:47:.. method:: list.pop([i])
Doc/library/array.rst:193:.. method:: array.pop([i])
Doc/library/stdtypes.rst:1116:| ``s.pop([i])``   | retrieves the 
item at *i* and  | \(2)|
```
the mutable sequence and array `pop()` method is documented as shown above in a 
way that implies the argument to `pop()` is a slice or sequence when it is 
actually just an integer. All those references should be `pop(i)` rather than 
`pop([i])`.

--
assignee: docs@python
components: Documentation
messages: 392551
nosy: docs@python, msapiro
priority: normal
severity: normal
status: open
title: Doc for mutable sequence pop() method implies argument is a slice or 
sequence.
type: behavior
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43996>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-30 Thread Mark Shannon


Mark Shannon  added the comment:

I'm claiming that this is fixed.

Feel free to reopen if the additional line in tracing causes a problem.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-30 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 5979e81a212949c62c2490167c9137d233d7de64 by Mark Shannon in 
branch 'master':
bpo-43933:  Set frame.f_lineno during call to __exit__ (GH-25719)
https://github.com/python/cpython/commit/5979e81a212949c62c2490167c9137d233d7de64


--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-04-30 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24440
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25729

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15443] datetime module has no support for nanoseconds

2021-04-30 Thread Mark Dickinson


Mark Dickinson  added the comment:

[Alexander]

> Is there high enough demand for nanoseconds in datetime and time instances?

One need that we've encountered in real code is simply for compatibility. We 
have Python code that interacts with a logging web service whose timestamps 
include nanosecond information. Whether or not nanosecond resolution makes 
sense for those timestamps is a moot point: that's out of our control.

When representing information retrieved from that web service in Python-land, 
we have a problem. If datetime.datetime had nanosecond precision, then using 
datetime.datetime to represent the retrieved values would be a no-brainer. As 
it is, we face a choice between:

- truncating or rounding to microsecond precision, and losing information as a 
result (which is particularly problematic if we want to write records back at 
any point)
- representing in some indirect form (as a str, an integer number of 
nanoseconds, a (datetime, nanoseconds) tuple, ...) and requiring the user to 
convert values for plotting or other analysis
- writing our own non-standard ns-supporting datetime class, or looking for a 
3rd party library with that support

None of those choices are terrible, but none of them are particularly palatable 
compared with using a standard library solution. (FWIW, we went with option 2, 
returning nanoseconds since the Unix epoch as an int.)

--

___
Python tracker 
<https://bugs.python.org/issue15443>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43977] Implement the latest semantics for PEP 634 for matching collections

2021-04-30 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 069e81ab3da46c441335ca762c4333b7bd91861d by Mark Shannon in 
branch 'master':
bpo-43977: Use tp_flags for collection matching (GH-25723)
https://github.com/python/cpython/commit/069e81ab3da46c441335ca762c4333b7bd91861d


--

___
Python tracker 
<https://bugs.python.org/issue43977>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-29 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 088a15c49d99ecb4c3bef93f8f40dd513c6cae3b by Mark Shannon in 
branch 'master':
bpo-43933: Show frame.f_lineno as None, rather than -1, if there is no line 
number. (GH-25717)
https://github.com/python/cpython/commit/088a15c49d99ecb4c3bef93f8f40dd513c6cae3b


--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43977] Implement the latest semantics for PEP 634 for matching collections

2021-04-29 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24415
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25723

___
Python tracker 
<https://bugs.python.org/issue43977>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43977] Implement the latest semantics for PEP 634 for matching collections

2021-04-29 Thread Mark Shannon


New submission from Mark Shannon :

PEP 634 has been updated to allow a faster and more robust implementation of 
matching sequences and mappings: https://github.com/python/peps/pull/1937

It needs to be implemented.

--
assignee: Mark.Shannon
messages: 392330
nosy: Mark.Shannon
priority: high
severity: normal
status: open
title: Implement the latest semantics for PEP 634 for matching collections

___
Python tracker 
<https://bugs.python.org/issue43977>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-04-29 Thread Mark Shannon


Mark Shannon  added the comment:

I've played around with a few formats, and what I've ended up with is this:

1. Use the >> marker for for exception targets, as well as normal branch 
targets.
2. Add a text version of the exception handler table at the end of the 
disassembly.

This has all the information, without too much visual clutter.

The function `f` above looks like this:
>>> dis.dis(f)
  2   0 NOP

  3   2 LOAD_CONST   1 (1)
  4 LOAD_CONST   2 (0)
  6 BINARY_TRUE_DIVIDE
  8 POP_TOP
 10 NOP
 12 LOAD_CONST   0 (None)
 14 RETURN_VALUE
>>   16 NOP
 18 PUSH_EXC_INFO

  4  20 POP_TOP
 22 POP_TOP
 24 POP_TOP

  5  26 NOP
 28 POP_EXCEPT
 30 LOAD_CONST   3 ('fail')
 32 RETURN_VALUE
>>   34 POP_EXCEPT_AND_RERAISE
ExceptionTable:
  2 to 8 -> 16 (depth 0)
  18 to 24 -> 34 (depth 3) lasti


The 'lasti' field indicates that the offset of the last instruction is pushed 
to the stack, which is needed for cleanup-then-reraise code.

--

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-04-29 Thread Mark Shannon


Mark Shannon  added the comment:

Responding to Serhiy's suggestions:
1 Add another column:
   Adding another column makes for lots of repetition in larger try blocks, and 
pushes useful information further to the right.
2 Add pseudo-instructions
  I find those misleading, as they aren't really there, and probably won't even 
correspond to the original SETUP_XXX instructions.

--

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-04-29 Thread Mark Shannon


Mark Shannon  added the comment:

> BTW, what are the three POP_TOP op codes in a row popping?

When exceptions are pushed to the stack, they are pushed as a triple: (exc, 
type, traceback)
so when we pop them, we need three pops.

--

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42739] Crash when try to disassemble bogus code object

2021-04-29 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42739>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-29 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +24410
pull_request: https://github.com/python/cpython/pull/25719

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-29 Thread Mark Shannon


Mark Shannon  added the comment:

I'm going to set the line number for executing the `__exit__()` function as 
that of the `with` keyword.

This should give a sensible traceback, and not break coverage, as the with 
statement is already executed.
It will impact traces and profiles, which is why the __exit__ call was not 
given a line number before.

Anthony, Ned,
please let me know if you disagree.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-29 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24408
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25717

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42739] Crash when try to disassemble bogus code object

2021-04-29 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset c76da79b37d2bcbe575cc927ba0a9b7a9ce465db by Mark Shannon in 
branch 'master':
bpo-42739: Don't use sentinels to mark end of line table. (GH-25657)
https://github.com/python/cpython/commit/c76da79b37d2bcbe575cc927ba0a9b7a9ce465db


--

___
Python tracker 
<https://bugs.python.org/issue42739>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43966] F String bugs with numpy.float32

2021-04-28 Thread Mark Dickinson


Mark Dickinson  added the comment:

See https://github.com/numpy/numpy/issues/10645. I'll close here (this tracker 
is for core Python, and NumPy isn't part of core Python).

--
nosy: +mark.dickinson
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43966>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43950] Include column offsets for bytecode instructions

2021-04-27 Thread Mark Shannon


Mark Shannon  added the comment:

The additional cost will not only be the line number table, but we need to 
store the line for exceptions that are reraised after cleanup.
Adding a column will mean more stack consumption.

--

___
Python tracker 
<https://bugs.python.org/issue43950>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42739] Crash when try to disassemble bogus code object

2021-04-27 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +24348
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25657

___
Python tracker 
<https://bugs.python.org/issue42739>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42739] Crash when try to disassemble bogus code object

2021-04-27 Thread Mark Shannon


Mark Shannon  added the comment:

Using sentinels as a marker to terminate the line number table, might be a 
problem if we want to use a different format. So I'm fixing this for 3.10.

--

___
Python tracker 
<https://bugs.python.org/issue42739>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32891] Add 'Integer' as synonym for 'Integral' in numbers module.

2021-04-26 Thread Mark Dickinson


Mark Dickinson  added the comment:

Sorry, Sergey: I don't have the time, energy or inclination for an extended 
discussion here, so I'll simply record my -1 for the proposed change and move 
on. If you want to push this change through, the way forward would likely be to 
find a core developer (maybe Terry?) who's willing to champion the change (and 
review the PR, of course).

--

___
Python tracker 
<https://bugs.python.org/issue32891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-26 Thread Mark Shannon


Mark Shannon  added the comment:

This test case was added for 3.10.
In 3.9 it reports that the pass statement executes, even though it is 
unreachable.

https://github.com/python/cpython/blob/master/Lib/test/test_sys_settrace.py#L919

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-26 Thread Mark Shannon


Mark Shannon  added the comment:

In this case, if happens that line 10 was the last line of code executing in 
the body of the with statement.
But the reason it was listed, was that it was statically the last line of code 
in the body.

If the body is something like:

1. if TRUE:
2. ...
3. else:
4. ...

Then the reported last line will be 4, even though it is never executed.
We can fix this, at the cost of extra compiler complexity and increased 
bytecode size for the non-exceptional case, but it is very difficult for the 
exceptional case, because we don't know statically where the exception has come 
from.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-26 Thread Mark Shannon


Mark Shannon  added the comment:

What exactly is the issue here?
That the line number is -1, or that is different from 3.9?

The `-1` should be `None`. I can fix that, but there is the issue of whether 
the cleanup code in a with statement should have a line number or not.

The traceback for 3.9 is incorrect, the program is not on line 10 when __exit__ 
is called.

Currently we mark the cleanup code for the with statement as artificial meaning 
that it has no line number.
We could give it the line number of the `with` keyword, but that might break 
coverage and profiling tools.

PEP 626 says that the "with" keyword is executable, and should be traced, but 
nothing about whether that includes the cleanup call to __exit__.

--
nosy: +nedbat

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43940] int casting to float results to a different value in memory

2021-04-26 Thread Mark Dickinson


Mark Dickinson  added the comment:

> Is there any workaround?

Sorry, I can't answer that without knowing what it is you're trying to do - 
that is, I don't know which part of the behaviour you want a workaround for. 
But that's getting off topic for this tracker, which is for bug reporting; I'd 
suggest taking the discussion to one of the mailing lists, Stack Overflow, or 
some other resource.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43940>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43940] int casting to float results to a different value in memory

2021-04-26 Thread Mark Dickinson


Change by Mark Dickinson :


--
resolution:  -> not a bug
status: open -> pending

___
Python tracker 
<https://bugs.python.org/issue43940>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43940] int casting to float results to a different value in memory

2021-04-26 Thread Mark Dickinson


Mark Dickinson  added the comment:

There's an implicit float-to-double conversion here (`fp.contents.value` is 
represented as a Python float, which is backed by a C double). What you're 
seeing is that converting the single-precision signaling NaN to double 
precision loses the signaling bit, producing a quiet double-precision NaN with 
the same sign and payload (modulo the signaling bit). That's what I'd expect 
under IEEE 754 semantics. (IEEE 754-2008 says, in section 6.2, "Under default 
exception handling, any operation signaling an invalid operation exception and 
for which a floating-point result is to be delivered shall deliver a quiet 
NaN.".)

I don't think there's anything we can or should do about this at Python level. 
ctypes is correctly reflecting the existing C-level behaviour. In other words, 
this is expected behaviour, not a bug.

--

___
Python tracker 
<https://bugs.python.org/issue43940>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43932] f-string decimal has leading space

2021-04-24 Thread Mark Dickinson


Change by Mark Dickinson :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43932>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43932] f-string decimal has leading space

2021-04-24 Thread Mark Dickinson


Mark Dickinson  added the comment:

This is by design, and documented under the "sign" section here: 
https://docs.python.org/3/library/string.html#format-specification-mini-language

The space before ".2f" is an instruction to leave space for a sign, "-" for a 
negative number and " " for a positive number. To avoid that space, use a 
format of ".2f" instead of " .2f".

>>> format(15, ".2f")
'15.00'
>>> format(15, " .2f")
' 15.00'
>>> format(15, "+.2f")
'+15.00'

--
nosy: +mark.dickinson

___
Python tracker 
<https://bugs.python.org/issue43932>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32891] Add 'Integer' as synonym for 'Integral' in numbers module.

2021-04-23 Thread Mark Dickinson


Mark Dickinson  added the comment:

For the record, here's the python-ideas thread: 
https://mail.python.org/archives/list/python-id...@python.org/thread/AWNTQZ3HHAQ42O2ZHRA3U2US4HGKC2TF/

I'm no longer convinced that this is a good idea. It's the usual difference 
between "Would it have been better if we'd called it 'Integer' rather than 
'Integral'?", and "Given where we are, is it worth changing the name now?". The 
name "Integral" isn't actually _wrong_ as such; it's just perhaps not the name 
that we would have chosen if we were inventing the numbers ABC right now.

If we add "Integer", we have to choose between

- maintaining both names indefinitely, or
- deprecating and eventually removing the "Integral" name

Neither option seems appealing: the first is a violation of "one obvious way"; 
the second causes unnecessary work for third-party projects already using 
Integral.

--
nosy: +rhettinger, tim.peters

___
Python tracker 
<https://bugs.python.org/issue32891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43475] Worst-case behaviour of hash collision with float NaN

2021-04-22 Thread Mark Dickinson


Mark Dickinson  added the comment:

Opened https://github.com/numpy/numpy/issues/18833

--

___
Python tracker 
<https://bugs.python.org/issue43475>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43475] Worst-case behaviour of hash collision with float NaN

2021-04-22 Thread Mark Dickinson


Mark Dickinson  added the comment:

Thanks, Raymond. I'll open something on the NumPy bug tracker, too, since they 
may want to consider doing something similar for numpy.float64 and friends.

--

___
Python tracker 
<https://bugs.python.org/issue43475>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43908] array.array should remain immutable

2021-04-22 Thread Mark Shannon


Mark Shannon  added the comment:

Adding an extra flag seems like the sensible thing to do for 3.10

Longer term, we should decouple immutability from whether something is a heap 
type.
I don't know why we would care that it is a heap type at all. Which bit of 
memory it happens to sit in seems irrelevant to anything but the GC.

Erland, could you turn your patch into PR?

--

___
Python tracker 
<https://bugs.python.org/issue43908>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27763] Add complex case to test_builtin abs()

2021-04-22 Thread Mark Dickinson


Change by Mark Dickinson :


--
resolution:  -> out of date

___
Python tracker 
<https://bugs.python.org/issue27763>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27763] Add complex case to test_builtin abs()

2021-04-22 Thread Mark Dickinson


Mark Dickinson  added the comment:

Yes, it looks as though we do have tests of the form that Evelyn describes in 
test_complex (which also seems like the right place for those tests).

Closing here. Thanks Sergey for the ping!

--
stage: test needed -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue27763>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43910] cgi.parse_header does not handle escaping correctly

2021-04-22 Thread Mark Gordon


Change by Mark Gordon :


--
keywords: +patch
pull_requests: +24236
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25519

___
Python tracker 
<https://bugs.python.org/issue43910>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43910] cgi.parse_header does not handle escaping correctly

2021-04-22 Thread Mark Gordon


New submission from Mark Gordon :

cgi.parse_header incorrectly handles unescaping of quoted-strings

Note that you can find the related RFCs to how HTTP encodes the Content-Type 
header at https://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html and further 
discussion on how quoted-string is defined at 
https://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-16.html#rfc.section.3.2.1.p.3.

The way parse_header is written it has no context to be able to tell if a 
backslash is escaping a double quote or if the backslash is actually the 
escaped character and the double quote is free-standing, unescaped. For this 
reason it fails to parse values that have a backslash literal at the end. e.g. 
the following Content-Type will fail to be parsed

a/b; foo="moo\\"; bar=baz

Example run on current cpython master demonstrating the bug:

Python 3.10.0a7+ (heads/master:660592f67c, Apr 21 2021, 22:51:04) [GCC 9.3.0] 
on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cgi
>>> query = 'a; foo="moo"; bar=cow' 
>>> print(query)
a; foo="moo\\"; bar=cow
>>> cgi.parse_header(query)
('a', {'foo': '"moo"; bar=cow'})

--
components: Library (Lib)
messages: 391580
nosy: msg555
priority: normal
severity: normal
status: open
title: cgi.parse_header does not handle escaping correctly
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue43910>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2021-04-21 Thread Mark Shannon


Mark Shannon  added the comment:

The changes to pyc format aren't user visible so shouldn't matter,
but what about the dis output?

Consider this program:

def f():
try:
1/0
except:
return "fail"

Currently it compiles to:
  2   0 SETUP_FINALLY7 (to 16)

  3   2 LOAD_CONST   1 (1)
  4 LOAD_CONST   2 (0)
  6 BINARY_TRUE_DIVIDE
  8 POP_TOP
 10 POP_BLOCK
 12 LOAD_CONST   0 (None)
 14 RETURN_VALUE

  4 >>   16 POP_TOP
 18 POP_TOP
 20 POP_TOP

  5  22 POP_EXCEPT
 24 LOAD_CONST   3 ('fail')
 26 RETURN_VALUE

With zero-cost exception handling, it will compile to something like:  
  2   0 NOP
  3   2 LOAD_CONST   1 (1)
  4 LOAD_CONST   2 (0)
  6 BINARY_TRUE_DIVIDE
  8 POP_TOP
 10 LOAD_CONST   0 (None)
 12 RETURN_VALUE

  None   14 PUSH_EXCEPT

  4  16 POP_TOP
 18 POP_TOP
 20 POP_TOP

  5  22 POP_EXCEPT
 24 LOAD_CONST   3 ('fail')
 26 RETURN_VALUE

(There are additional optimizations that should be applied, but those are a 
separate issue)

The problem is that the exception handling flow is no longer visible.
Should we add it back in somehow, or just append the exception jump table?

--

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



<    6   7   8   9   10   11   12   13   14   15   >