[issue45093] Add method to compare dicts accounting for order

[Michael Rans]

Michael Rans  added the comment:

Thanks for all your help and advice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45093] Add method to compare dicts accounting for order

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> Another case is in round trip processing of JSON or YML.

Sorry for my ignorance, but I don't see how or why an unordered 
comparison would help you with round-tripping JSON or YAML.

The order of key:value pairs in JSON is not guaranteed to be preserved, 
so if you round-trip a dict to JSON back to a dict, and then use an 
ordered comparison, you might wrongly think that they are unequal.

(I think that Python's JSON does preserve order, by default. But other 
JSON encoders might not.)

> Other cases are where you would prefer an OrderedDict over a dict. 

Then use an OrderedDict.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41010] email.message.EmailMessage.get_body

[Andrei Kulakov]

Change by Andrei Kulakov :


--
nosy: +kj
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type: crash -> behavior
versions: +Python 3.11 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45086] f-string unmatched ']'

[Eric V. Smith]

Eric V. Smith  added the comment:

I don't think it really makes a difference, but here's some background:

For f-strings, the parser itself does not break apart the f-string into 
(, ) parts. There's a custom parser (at 
https://github.com/python/cpython/blob/0b58e863df9970b290a4de90c67f9ac30c443817/Parser/string_parser.c#L837)
 which does that. Then the normal parser is used to parse the expression 
portion.

I think the error shown here is not in the expression parser, but in the 
fstring parser in fstring_find_expr(), at 
https://github.com/python/cpython/blob/0b58e863df9970b290a4de90c67f9ac30c443817/Parser/string_parser.c#L665

As Terry says, it's not incorrect to print the error show in this bug report.

To further diverge:

There's been talk about using the normal parser to pull apart the entire 
f-string, instead of using the two-pass version I mention above. But we've 
never gotten past just talking about it. There are pros and cons for doing it 
with the normal parser, but that's a discussion for a different forum.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44815] asyncio.gather no DeprecationWarning if task are passed

[Chih-Hsuan Yen]

Chih-Hsuan Yen  added the comment:

A regression in 3.9.7 (issue45097) seems related to this issue. Could you have 
a look?

--
nosy: +yan12125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45097] "The loop argument is deprecated" reported when user code does not use it

[Chih-Hsuan Yen]

New submission from Chih-Hsuan Yen :

With Python 3.9.7, "DeprecationWarning: The loop argument is deprecated" may be 
reported when user code does not use it. Here is an example:

import asyncio
import warnings

warnings.filterwarnings('error')

def crash():
raise KeyboardInterrupt

async def main():
asyncio.get_event_loop().call_soon(crash)
await asyncio.sleep(5)

try:
asyncio.run(main())
except KeyboardInterrupt:
pass

On 3.9.6, no warning is reported, while results on 3.9.7 are

Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 629, in 
run_until_complete
self.run_forever()
  File "/usr/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
self._run_once()
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
handle._run()
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
  File 
"/home/yen/var/local/Computer/archlinux/community/python-anyio/trunk/cpython-3.9.7-regression.py",
 line 11, in crash
raise KeyboardInterrupt
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/home/yen/var/local/Computer/archlinux/community/python-anyio/trunk/cpython-3.9.7-regression.py",
 line 18, in 
asyncio.run(main())
  File "/usr/lib/python3.9/asyncio/runners.py", line 47, in run
_cancel_all_tasks(loop)
  File "/usr/lib/python3.9/asyncio/runners.py", line 64, in _cancel_all_tasks
tasks.gather(*to_cancel, loop=loop, return_exceptions=True))
  File "/usr/lib/python3.9/asyncio/tasks.py", line 755, in gather
warnings.warn("The loop argument is deprecated since Python 3.8, "
DeprecationWarning: The loop argument is deprecated since Python 3.8, and 
scheduled for removal in Python 3.10.

As indicated by the traceback, the loop argument is used inside the asyncio 
library, not from user code. It has been an issue for some time, and the issue 
is exposed after changes for issue44815.

Credit: this example code is modified from an anyio test 
https://github.com/agronholm/anyio/blob/3.3.0/tests/test_taskgroups.py#L943. I 
noticed this issue when I was testing anyio against 3.9.7.

--
components: asyncio
messages: 401032
nosy: asvetlov, yan12125, yselivanov
priority: normal
severity: normal
status: open
title: "The loop argument is deprecated" reported when user code does not use it
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45093] Add method to compare dicts accounting for order

[Terry J. Reedy]

Terry J. Reedy  added the comment:

-1 also for the reasons given.

Michael, a question asking how would have made a good python-list post. So I 
posted the answers in "How to include insertion order in dict equality"

--
nosy: +terry.reedy
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45093] Add method to compare dicts accounting for order

[Raymond Hettinger]

Change by Raymond Hettinger :


--
Removed message: https://bugs.python.org/msg401022

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45093] Add method to compare dicts accounting for order

[Raymond Hettinger]

Raymond Hettinger  added the comment:

-1 The dict API is too important to be burdened with a mostly useless and 
rarely needed method.  Besides we already have simple ways to do it, for 
example:

assert list(d.items()) == list(e.items())

or:

assert OrderedDict(d) == OrderedDict(e)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45086] f-string unmatched ']'

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The behavior remains the same in 3.11 ('main' branch).  New PEG parser parses 
this the same.

(Pablo, if there is a mistake below, please correct.)

Normally, the parser copies code chars between quotes, with or without '\' 
interpretation, into a string object.  When the 'f' prefix is given, the string 
gets divided into substrings and replacement fields.  The code part of each 
replacement field is separately parsed.  There are two options: 1. parse the 
field code while looking for an endcode marker; 2. look ahead for an endcode 
marker and then parse the code.

The current behavior is consistent with opotion 1 and the python policy of 
reporting the first error found and exiting, rather than trying to 
resynchronize to try to find more errors.

--
nosy: +pablogsal, terry.reedy
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
versions: +Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45060] Do not use the equality operators with None

[Terry J. Reedy]

Change by Terry J. Reedy :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45060] Do not use the equality operators with None

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset e2b29304137e6253b7bb89c180ef5d113d60b4eb by Serhiy Storchaka in 
branch '3.10':
[3.10] bpo-45060: Get rid of few uses of the equality operators with None 
(GH-28087). (GH-28092)
https://github.com/python/cpython/commit/e2b29304137e6253b7bb89c180ef5d113d60b4eb


--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Nick Coghlan]

Nick Coghlan  added the comment:

For the module metadata problem: one potential approach to that for "designed 
to be frozen" stdlib modules is to set the values directly in the module code, 
rather than trying to set them automatically in the frozen import machinery.

It should also be possible to delete the implicitly created metadata fields and 
use module level dynamic attribute retrieval to find the stdlib source code for 
tracebacks and introspection purposes without incurring any start up costs.

--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45058] Undefined behavior for syntax "except AError or BError:" accepted by interpreter

[Terry J. Reedy]

Terry J. Reedy  added the comment:

"except a or b:" should be same as "except (a or b):" which should be same as 
"except a:", which is current behavior in 3.10.0, etc.

--
nosy: +terry.reedy
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Guido van Rossum]

Guido van Rossum  added the comment:

Let's wait until Mark Shannon is back from vacation (another week).

Note that class docstrings *are* contained in the class body code object -- 
there's executable code equivalent to

__doc__ = "this is the docstring"

But I agree it's not easily found without analyzing the bytecode.

Maybe the status quo is best after all? I would like to be able to identify 
code objects for functions, we could add a bit to co_flags for that.

--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Inada Naoki]

Inada Naoki  added the comment:

> I am still not convinced that it's a good idea to put the docstring in the 
> surrounding code object. I'd like to be able to see it when I introspect a 
> code object, not just when introspecting a function object (I may be 
> analyzing code only, and it's hard to connect the code object with the 
> NEW_FUNCTION opcode in the parent code object -- you have to scan the 
> bytecode, which is fragile.)

I think that reasoning is not strong enough to add new member to code object.

* Modules and classes don't get docstring from their code objects. Why only 
functions need to store docstring?
* Lambdas, comprehensions, and PEP 649 (if acceptted) uses code objects but no 
docstring. Why they need to pay cost of `co_doc` member? (cost = memory + 
unmarshal time).

Code objects have filename and firstlineno. And there are many functions 
without docstring. So removing docstring from code object won't make inspection 
hard so much.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Guido van Rossum]

Guido van Rossum  added the comment:

Since we're not changing the magic number, wheels created for rc1 will still 
work with the final 3.10 (and vice versa!) -- creating a wheel or PYC file just 
will produce a different sequence of bytes, but there are other things that do 
that.

Then again, since we're not changing the magic number, it's not the end of the 
world to put off the backport to 3.10.1.

So I've convinced myself that it actually doesn't matter, and probably it's 
best to wait for 3.10.1 (changes to the compiler are always risky).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45093] Add method to compare dicts accounting for order

[Raymond Hettinger]

Raymond Hettinger  added the comment:

-1 The dict API is to important to be burdened with a mostly useless and rarely 
needed method.  Besides we already have simple ways to do it, for example:

assert list(d.items()) == list(e.items())

or:

assert OrderedDict(d) == OrderedDict(e)

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I honestly want to backport it because I think it should not have any negative 
impact, but on the other hand I don't feel very confident as the release 
candidate phase is supposed to be as close as possible as the final release and 
this is not fixing a critical bug. The devguide says about the RC:


>>Generally, these issues must be severe enough (e.g. crashes) that they 
>>deserve fixing before the final release. All other issues should be deferred 
>>to the next development cycle, since stability is the strongest concern at 
>>this point.

I am just trying to be cautious but on the other hand we still have anltbet 
release candidate for people to try it out before the final release so if you 
all think is better to have this on the RC and this is not going to be an issue 
for existing universal wheels and the like, then I suppose we can merge it 
before Rc2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45034] Improve struct.pack out of range error messages

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I agree, including use of hex, if possible, for unsigned (non-negative) values.

Grepping 'format requires' returns
F:\dev\3x\Modules\_struct.c: 365: "'%c' format requires 0 <= number 
<= %zu",
F:\dev\3x\Modules\_struct.c: 371: "'%c' format requires %zd <= 
number <= %zd",
F:\dev\3x\Modules\_struct.c: 550: "byte format requires 
-128 <= number <= 127");
F:\dev\3x\Modules\_struct.c: 565: "ubyte format 
requires 0 <= number <= 255");
F:\dev\3x\Modules\_struct.c: 577: "char format requires 
a bytes object of length 1");
F:\dev\3x\Modules\_struct.c: 593: "short format 
requires " Py_STRINGIFY(SHRT_MIN)
F:\dev\3x\Modules\_struct.c: 611: "ushort format 
requires 0 <= number <= "
   
Py_STRINGIFY(USHRT_MAX));

I believe l365 is the source for the 2nd example.  AFAIK, 'zu' is not valid for 
either C printf or Python % formating.
Lines 611 and 612 are the source for the 1st example.  From comments before 
line 365, there can be issues with lefts shifts, but '0x', '0x_', 
and '0x___' could be hard-coded strings that would cover all 
'normal' systems.

Grepping "argument out of range" returns
F:\dev\3x\Modules\_struct.c: 168: "argument out of 
range");
F:\dev\3x\Modules\_struct.c: 192: "argument out of 
range");
F:\dev\3x\Modules\_struct.c: 215: "argument out of 
range");
F:\dev\3x\Modules\_struct.c: 238: "argument out of 
range");
F:\dev\3x\Modules\_struct.c: 261: "argument out of 
range");
F:\dev\3x\Modules\_struct.c: 284: "argument out of 
range");

It is nnclear to me without more reading why some codes lead to this less 
helpful message.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.11 -Python 2.7, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25291] better Exception message for certain task termination scenario

[Irit Katriel]

Irit Katriel  added the comment:

This is doing something different now (on 3.11):

iritkatriel@Irits-MBP cpython % ./python.exe ttt.py
/Users/iritkatriel/src/cpython/ttt.py:5: DeprecationWarning: There is no 
current event loop
  loop = asyncio.get_event_loop()
i am task..

[here it hangs and I kill it with ctrl-C]

^CTraceback (most recent call last):
  File "/Users/iritkatriel/src/cpython/ttt.py", line 12, in 
loop.run_forever()
^^
  File "/Users/iritkatriel/src/cpython/Lib/asyncio/base_events.py", line 595, 
in run_forever
self._run_once()

  File "/Users/iritkatriel/src/cpython/Lib/asyncio/base_events.py", line 1841, 
in _run_once
event_list = self._selector.select(timeout)
 ^^
  File "/Users/iritkatriel/src/cpython/Lib/selectors.py", line 562, in select
kev_list = self._selector.control(None, max_ev, timeout)
   ^
KeyboardInterrupt
Task exception was never retrieved
future:  exception=RuntimeError('Task does not 
support set_exception operation')>
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython/ttt.py", line 8, in task
myself.set_exception(RuntimeError('something bad'))
^^^
RuntimeError: Task does not support set_exception operation

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23354] Loading 2 GiLOC file which raises exception causes wrong traceback

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: The sqlite3 timestamp conversion between unixepoch and localtime

[Barry]

> On 3 Sep 2021, at 13:40, Bob Martin  wrote:
> 
> On 2 Sep 2021 at 20:25:27, Alan Gauld  wrote:
>> On 02/09/2021 20:11, MRAB wrote:
>> 
 In one of them (I can't recall which is which) they change on the 4th
 weekend of October/March in the other they change on the last weekend.
 
 
>>> In the EU (and UK) it's the last Sunday in March/October.
>>> 
>>> In the US it's second Sunday in March and the first Sunday in November.
>>> 
>>> I know which one I find easier to remember!
>> 
>> Interesting. I remember it as closer than that. The bugs we found were
>> due to differences in the DST settings of the BIOS in the PCs. (They
>> were deliberately all sourced from DELL but the EU PCs had a slightly
>> different BIOS).
>> 
>> The differences you cite should have thrown up issues every year.
>> I must see if I can find my old log books...
>> 
> 
> ISTR that the USA changes were the same as the EU until a few years ago.

I recall that DST changes have been at least 1 week different between the UK 
and USA since the 80’s.

Barry
> 
> I remember thinking at the time it changed "why would they do that?"
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Sam Bull]

Change by Sam Bull :


--
nosy: +dreamsorcerer
nosy_count: 8.0 -> 9.0
pull_requests: +26587
pull_request: https://github.com/python/cpython/pull/28149

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45075] confusion between frame and frame_summary in traceback module

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45075] confusion between frame and frame_summary in traceback module

[Irit Katriel]

Irit Katriel  added the comment:


New changeset 0b58e863df9970b290a4de90c67f9ac30c443817 by Irit Katriel in 
branch 'main':
bpo-45075: distinguish between frame and FrameSummary in traceback mo… 
(GH-28112)
https://github.com/python/cpython/commit/0b58e863df9970b290a4de90c67f9ac30c443817


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Guido van Rossum]

Guido van Rossum  added the comment:

Are you sure? I know we're really close to the release, but I'd much rather
change the PYC format during the RC cycle than once we're doing bugfix
releases. I don't think we've ever changed the magic number between bugfix
releases. (I realize this doesn't change the magic number, but it's
probably also the first time we make a change that affects marshal output
without requiring changes in marshal input.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45053] MD5SumTests.test_checksum_fodder fails on Windows

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Yes, it was encodings problem :)

This line solved it (here: 
https://github.com/python/cpython/blob/6f8bc464e006f672d1aeafbfd7c774a40215dab2/Tools/scripts/md5sum.py#L69):

```python
out.write('%s %s\n' % (m.hexdigest(), filename.encode(
sys.getfilesystemencoding(),
).decode(sys.stdout.encoding)))
```

> The simplest way to "fix" the test is using TESTFN_ASCII instead of TESTFN.

I haven't changed this, because right now it should work for non-ASCII symbols 
as well. I can even add an explicit ASCII test if needed.

Shouldn't https://github.com/python/cpython/pull/28060 be merge before I submit 
a new PR, so we can be sure that test now works? In the current state it will 
be just ignored.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45096] Update Tools/freeze to make use of Tools/scripts/freeze_modules.py?

[Eric Snow]

New submission from Eric Snow :

In bpo-45019 we added Tools/scripts/freeze_modules.py to improve how we manage 
which modules get frozen by default.  (We turned previously manual steps into 
automation of generated code.)  There is probably some overlap with what we do 
in Tools/freeze/freeze.py.  Is so, we should make changes for better re-use.

--
components: Demos and Tools
messages: 401015
nosy: eric.snow, lemburg
priority: normal
severity: normal
stage: needs patch
status: open
title: Update Tools/freeze to make use of Tools/scripts/freeze_modules.py?
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[Brandt Bucher]

Change by Brandt Bucher :


--
pull_requests: +26586
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/28147

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45093] Add method to compare dicts accounting for order

[Michael Rans]

Michael Rans  added the comment:

Thank you. Another case is in round trip processing of JSON or YML. Other cases 
are where you would prefer an OrderedDict over a dict. 

I think the method would help clarify things because it would make it obvious 
that it is for ordered comparisons while the existing == does not do that.

eg. something like:
d1.compare_ordered(d2)
or:
d1.compare(d2, ordered=True)  # ordered could be by default True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[Python-announce] ANN: pvlib-0.9.0 released

[Dr. Mark Alexander Mikofski PhD]

  Dear Pythonistas and solar power enthusiasts,

On behalf of the maintainers, we're happy to announce a new release of
pvlib python:
software for simulating performance of photovoltaic solar energy systems.

*See what's new for v0.9.0:*
* https://pvlib-python.readthedocs.io/en/stable/whatsnew.html

*Releases are available from PyPI and the conda-forge channel:*
* https://pypi.org/project/pvlib/
* https://anaconda.org/conda-forge/pvlib-python

*Read the Documentation:*
* https://pvlib-python.readthedocs.io/en/stable/index.html

*Report issues & contribute:*
* https://github.com/pvlib/pvlib-python

*Highlights:*
* There are some breaking changes
,
so please read the Release Notes carefully before updating.
* pvlib now has a new Array class

that
represents groups of identical modules that are part of a PVSystem but have
the same orientation and mounting type.
* The PVSystem class

can
now model a combination of arrays each with different orientations,
modules, and mounting types.
* pvlib completed a very successful GSoC
, which added several new iotools
like get_bsrn()

, get_pvgis_hourly()

, get_cams()
,
and more. Special thanks to Adam R. Jensen ,
Kevin Anderson, and many reviewers who helped!
* Many bug fixes

including
a fix to Perez  contributed
by Clean Power Research. Read their blog

to
learn more about how they're contributing to pvlib. Thanks!

*The maintainers thank you for using pvlib python!*


-- 
Mark Mikofski, PhD (2005)
*Fiat Lux*
___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

[issue45022] Update libffi to 3.4.2

[Steve Dower]

Steve Dower  added the comment:

So on one hand, this change applies cleanly and it appears nothing needs to 
change to adopt the newer version.

On the other hand, because the libffi DLL has a different name, it changes the 
layout on disk. I know we don't do that (except in exceptional circumstances) 
after beta, but perhaps this one is safe enough? We certainly don't "support" 
directly accessing the libffi DLL, but it still could break users (if, for 
example, they load another native module that implicitly used the older libffi 
and would now fail to load when it's missing).

Any thoughts?

--
stage: patch review -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45022] Update libffi to 3.4.2

[Steve Dower]

Steve Dower  added the comment:


New changeset 6f8bc464e006f672d1aeafbfd7c774a40215dab2 by Steve Dower in branch 
'main':
bpo-45022: Update libffi to 3.4.2 in Windows build (GH-28146)
https://github.com/python/cpython/commit/6f8bc464e006f672d1aeafbfd7c774a40215dab2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12499] textwrap.wrap: add control for fonts with different character widths

[Éric Araujo]

Éric Araujo  added the comment:

A PR was opened for this.  `text_len` is used as param/attribute name.

--
versions: +Python 3.11 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Add a method to list the current named logging levels

[Barry]

> On 2 Sep 2021, at 23:38, Dieter Maurer  wrote:
> 
> Edward Spencer wrote at 2021-9-2 10:02 -0700:
>> Sometimes I like to pass the logging level up to the command line params so 
>> my user can specific what level of logging they want. However there is no 
>> easy method for pulling the named logging level names.
>> 
>> Looking into the code, it would actually be incredibly easy to implement;
>> 
>> in `logging.__init__.py`;
>> 
>> def listLevelNames():
>>   return _nameToLevel.keys()
>> 
>> You could obviously add some other features, like listing only the defaults, 
>> sorted by numerical level or alphabetically, etc. But really this basic 
>> implementation would be enough to expose the internal variables which 
>> shouldn't be accessed because they change (and in fact, between python 2 and 
>> 3, they did).
>> 
>> Any thoughts?
> 
> Usually, you use 5 well known log levels: "DEBUG", "INFO", "WARNING",
> "ERROR" and "CRITICAL".
> No need to provide a special function listing those levels.

I add my own levels, but then I know I did it.

Barry

> 
> 
> 
> --
> Dieter
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically.

[Chris Angelico]

On Sat, Sep 4, 2021 at 3:33 AM Alan Gauld via Python-list
 wrote:
>
> On 02/09/2021 19:30, Chris Angelico wrote:
>
> >> Without DST the schools opened in the dark so all the kids
> >> had to travel to school in the dark and the number of
> >> traffic accidents while crossing roads jumped.
> >
> > How do they manage in winter?
>
> That was the winter. Sunrise wasn't till 10:00 or so
> and the schools open at 9. With DST sunrise became
> 9:00 and with pre-dawn light it is enough to see by.

Are you saying that you had DST in winter, or that, when summer *and*
DST came into effect, there was more light at dawn? Because a *lot* of
people confuse summer and DST, and credit DST with the natural effects
of the season change.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Connecting python to DB2 database

[Dennis Lee Bieber]

On Fri, 3 Sep 2021 09:29:20 -0400, DFS  declaimed the
following:

>
>Now can you get DB2 to accept ; as a SQL statement terminator like the 
>rest of the world?   They call it "An unexpected token"...

I've never seen a semi-colon used for SQL statements via a db-api
adapter. The semi-colon is, in my experience, only required by basic
interactive query utilities -- to tell the utility that the statement is
fully entered, and can be sent to the server.


-- 
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The sqlite3 timestamp conversion between unixepoch and localtime can't be done according to the timezone setting on the machine automatically.

[Alan Gauld via Python-list]

On 02/09/2021 19:30, Chris Angelico wrote:

>> Without DST the schools opened in the dark so all the kids
>> had to travel to school in the dark and the number of
>> traffic accidents while crossing roads jumped.
> 
> How do they manage in winter? 

That was the winter. Sunrise wasn't till 10:00 or so
and the schools open at 9. With DST sunrise became
9:00 and with pre-dawn light it is enough to see by.

Its a recurring theme. Every now and then some smart
young politician from the south of the UK suggests
doing away with DST and a large crowd of northerners
jump up and say no way!

> Can that be solved with better street> lighting?

They had street lighting but it casts dark shadows etc.
In fact modern LED based street lighting is worse in
that respect that the old yellow sodium lights were.
But where it doesn't exist the cost of installing
street lighting in small villages is too high compared
to just changing the clocks. And of course street
lighting has a real running cost that would be reflected
in the local council taxes, and nobody wants to
pay more of them! After all street lighting has
been available for over 150 years, if they haven't
installed it by now (I mean, nearly everywhere
has some lighting, at least on the main roads,
it's just the smaller back streets that tend to be dark.)

> That was fifty years ago now, and the negative consequences
> of DST are far stronger now.

But not apparent to most people. Most still see it
as a benefit because they get longer working daylight.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[Peter Pearson]

On Thu, 2 Sep 2021 07:54:27 -0700 (PDT), Julio Di Egidio wrote:
> On Thursday, 2 September 2021 at 16:51:24 UTC+2, Christian Gollwitzer wrote:
>> Am 02.09.21 um 16:49 schrieb Julio Di Egidio:
>> > On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote: 
>> >> On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote: 
>> > 
>> >>> 39.61 
>> >> 
>> >> Welcome to the exciting world of roundoff error: 
>> > 
>> > Welcome to the exiting world of Usenet. 
>> > 
>> > *Plonk*
>> 
>> Pretty harsh, isn't it? He gave a concise example of the same inaccuracy 
>> right afterwards. 
>
> And I thought you were not seeing my posts...
>
> Given that I have already given a full explanation, you guys, that you
> realise it or not, are simply adding noise for the usual pub-level
> discussion I must most charitably guess.
>
> Anyway, just my opinion.  (EOD.)


Although we are in the world of Usenet, comp.lang.python is by
no means typical of Usenet.  This is a positive, helpful, welcoming
community in which "Plonk", "EOD", and "RTFM" (appearing in another
post) are seldom seen, and in which I have never before seen the
suggestion that everybody else should be silent so that the silver
voice of the chosen one can be heard.


-- 
To email me, substitute nowhere->runbox, invalid->com.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The sqlite3 timestamp conversion between unixepoch and localtime

[Michael F. Stemper]

On 03/09/2021 01.14, Bob Martin wrote:

On 2 Sep 2021 at 20:25:27, Alan Gauld  wrote:

On 02/09/2021 20:11, MRAB wrote:


In one of them (I can't recall which is which) they change on the 4th
weekend of October/March in the other they change on the last weekend.



In the EU (and UK) it's the last Sunday in March/October.

In the US it's second Sunday in March and the first Sunday in November.

I know which one I find easier to remember!


Interesting. I remember it as closer than that. The bugs we found were
due to differences in the DST settings of the BIOS in the PCs. (They
were deliberately all sourced from DELL but the EU PCs had a slightly
different BIOS).

The differences you cite should have thrown up issues every year.
I must see if I can find my old log books...



ISTR that the USA changes were the same as the EU until a few years ago.

I remember thinking at the time it changed "why would they do that?"


It was part of the Energy Policy Act of 2005[1].

However, saying that doesn't explain "why".

The explanation given at the time was that it would save energy
because people wouldn't need to turn on their lights as early.
This ignored the fact that we needed to have them on later in
the morning.

The required studies were inconclusive, but some reported that
since it was light later in the day, people went driving around
in the evening, causing aggregate energy consumption to increase
rather than decrease.

One of the bill's sponsors said that having it be light later in
the day would "make people happy".

Talk at the time (which I never verified or refuted) said that he
got significant campaign contributions from a trade group for
outdoor cooking (grills, charcoal, usw) and that they wanted it
so that the grilling season would be longer, leading to more
revenue for them.

At the time, I was product manager for real-time control systems
for critical infrastructure. Having to collect the changes to
zoneinfo, whatever the equivalent file for Oracle was, revalidate
our entire system, and get information/patches to our North American
customers was annoying.


[1] , Sec 110

--
Michael F. Stemper
If you take cranberries and stew them like applesauce they taste much
more like prunes than rhubarb does.
--
https://mail.python.org/mailman/listinfo/python-list

[issue42255] webbrowser.MacOSX is unused, untested and undocumented

[Łukasz Langa]

Łukasz Langa  added the comment:

Thanks for the patches, Dong-hee Na! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42255] webbrowser.MacOSX is unused, untested and undocumented

[Łukasz Langa]

Change by Łukasz Langa :


--
versions: +Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42255] webbrowser.MacOSX is unused, untested and undocumented

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 2a8956c268772fd31aeeb6ee522f123af94a2926 by Dong-hee Na in branch 
'3.10':
bpo-42255: Update webbrowser doc for macOS (GH-28144)
https://github.com/python/cpython/commit/2a8956c268772fd31aeeb6ee522f123af94a2926


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42255] webbrowser.MacOSX is unused, untested and undocumented

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset ce83e42437b8e5a4bf4237f981a7a90401922456 by Dong-hee Na in branch 
'3.9':
bpo-42255: Update webbrowser doc for macOS (GH-28145)
https://github.com/python/cpython/commit/ce83e42437b8e5a4bf4237f981a7a90401922456


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Let's wait for 3.10.1 to backport this because I prefer to keep pyc files 
stable for the release candidate. Turns out that people are already preparing 
wheels to 3.10 and although this may be fine, I don't want to risk 
incompatibilities for the release.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33618] Support TLS 1.3

[Philip Prindeville]

Change by Philip Prindeville :


--
nosy: +philipp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45022] Update libffi to 3.4.2

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +26585
pull_request: https://github.com/python/cpython/pull/28146

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43669] PEP 644: Require OpenSSL 1.1.1 or newer

[Philip Prindeville]

Change by Philip Prindeville :


--
nosy: +philipp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45095] Easier loggers traversal tree with a logger.getChildren method

[Stéphane Blondon]

New submission from Stéphane Blondon :

Currently, logging.root.manager.loggerDict is usable to do a homemade traversal 
of the loggers tree. However, it's not a public interface. Adding a 
'logger.getChildren()' method would help to implement the traversal. The method 
would return a set of loggers.


Usage example:
>>> import logging
>>> logging.basicConfig(level=logging.CRITICAL)
>>> root_logger = logging.getLogger()

>>> root_logger.getChildren()
set()
>>> a_logger = logging.getLogger("a")
>>> root_logger.getChildren()
{}
>>> logging.getLogger('a.b').setLevel(logging.DEBUG)
>>> _ = logging.getLogger('a.c')
>>> a_logger.getChildren()
{, }


With such method, traverse the tree will be obvious to write with a recursive 
function.


Use cases:
 - to check all the loggers are setted up correctly. I wrote a small function 
to get all loggers, and log on every level to check the real behaviour.
 - to draw the loggers tree like logging_tree library 
(https://pypi.org/project/logging_tree/). I didn't ask to logging_tree's 
maintainer but I don't think he would use such function because the library 
works for huge range of python releases.


I plan to write a PR if someone thinks it's a good idea.

--
components: Library (Lib)
messages: 401006
nosy: sblondon
priority: normal
severity: normal
status: open
title: Easier loggers traversal tree with a logger.getChildren method
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Select columns based on dates - Pandas

[Martin Di Paola]

You may want to reshape the dataset to a tidy format: Pandas works 
better with that format.


Let's assume the following dataset (this is what I understood from your 
message):


In [34]: df = pd.DataFrame({
...: 'Country': ['us', 'uk', 'it'],
...: '01/01/2019': [10, 20, 30],
...: '02/01/2019': [12, 22, 32],
...: '03/01/2019': [14, 24, 34],
...: })

In [35]: df
Out[35]:
  Country  01/01/2019  02/01/2019  03/01/2019
0  us  10  12  14
1  uk  20  22  24
2  it  30  32  34

Then, reshape it to a tidy format. Notice how each row now represents 
a single measure.


In [43]: pd.melt(df, id_vars=['Country'], var_name='Date', 
value_name='Cases')

Out[43]:
  CountryDate  Cases
0  us  01/01/2019 10
1  uk  01/01/2019 20
2  it  01/01/2019 30
3  us  02/01/2019 12
4  uk  02/01/2019 22
5  it  02/01/2019 32
6  us  03/01/2019 14
7  uk  03/01/2019 24
8  it  03/01/2019 34

I used strings to represent the dates but it is much handy work
with real date objects.

In [44]: df2 = _
In [45]: df2['Date'] = pd.to_datetime(df2['Date'])

Now we can filter by date:

In [50]: df2[df2['Date'] < '2019-03-01']
Out[50]:
  Country   Date  Cases
0  us 2019-01-01 10
1  uk 2019-01-01 20
2  it 2019-01-01 30
3  us 2019-02-01 12
4  uk 2019-02-01 22
5  it 2019-02-01 32

With that you could create three dataframes, one per month.

Thanks,
Martin.


On Thu, Sep 02, 2021 at 12:28:31PM -0700, Richard Medina wrote:

Hello, forum,
I have a data frame with covid-19 cases per month from 2019 - 2021 like a 
header like this:

Country, 01/01/2019, 2/01/2019, 01/02/2019, 3/01/2019, ... 01/01/2021, 
2/01/2021, 01/02/2021, 3/01/2021

I want to filter my data frame for columns of a specific month range of march 
to September of 2019, 2020, and 2021 only (three data frames).

Any ideas?
Thank you


--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

[issue45083] Need to use the exception class qualname when rendering exception (in C code)

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 41c23740243cc3a0699bc4d5dcfd47a0007ff039 by Miss Islington (bot) 
in branch '3.9':
[3.9] bpo-45083: Include the exception class qualname when formatting an 
exception (GH-28119) (GH-28135)
https://github.com/python/cpython/commit/41c23740243cc3a0699bc4d5dcfd47a0007ff039


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45022] Update libffi to 3.4.2

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 8c3a10e58b12608c3759fee684e7aa399facae2a by Steve Dower in branch 
'3.8':
bpo-45022: Pin current libffi build to fixed version in preparation for 
upcoming update (GH-27982) (GH-28001)
https://github.com/python/cpython/commit/8c3a10e58b12608c3759fee684e7aa399facae2a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42255] webbrowser.MacOSX is unused, untested and undocumented

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +26584
pull_request: https://github.com/python/cpython/pull/28145

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Guido van Rossum]

Guido van Rossum  added the comment:

I am still not convinced that it's a good idea to put the docstring in the 
surrounding code object. I'd like to be able to see it when I introspect a code 
object, not just when introspecting a function object (I may be analyzing code 
only, and it's hard to connect the code object with the NEW_FUNCTION opcode in 
the parent code object -- you have to scan the bytecode, which is fragile.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42255] webbrowser.MacOSX is unused, untested and undocumented

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +26583
pull_request: https://github.com/python/cpython/pull/28144

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[Brandt Bucher]

Brandt Bucher  added the comment:

Found it. This particular build is configured with HAVE_ALIGNED_REQUIRED=1, 
which forces it to use fnv instead siphash24 as its string hashing algorithm.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42255] webbrowser.MacOSX is unused, untested and undocumented

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset bc1c49fa94b2abf70e6937373bf1e6b5378035c5 by Dong-hee Na in branch 
'main':
bpo-42255: Deprecate webbrowser.MacOSX from Python 3.11 (GH-27837)
https://github.com/python/cpython/commit/bc1c49fa94b2abf70e6937373bf1e6b5378035c5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1514420] Traceback display code can attempt to open a file named ""

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.11 -Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1514420] Traceback display code can attempt to open a file named ""

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +patch
pull_requests: +26582
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28143

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

RE: on floating-point numbers

[Schachner, Joseph]

Actually, Python has an fsum function meant to address this issue.

>>> math.fsum([1e14, 1, -1e14])
1.0
>>>

Wow it works.

--- Joseph S.

Teledyne Confidential; Commercially Sensitive Business Data

-Original Message-
From: Hope Rouselle  
Sent: Thursday, September 2, 2021 9:51 AM
To: python-list@python.org
Subject: on floating-point numbers

Just sharing a case of floating-point numbers.  Nothing needed to be solved or 
to be figured out.  Just bringing up conversation.

(*) An introduction to me

I don't understand floating-point numbers from the inside out, but I do know 
how to work with base 2 and scientific notation.  So the idea of expressing a 
number as 

  mantissa * base^{power}

is not foreign to me. (If that helps you to perhaps instruct me on what's going 
on here.)

(*) A presentation of the behavior

>>> import sys
>>> sys.version
'3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit 
(AMD64)]'

>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>> sum(ls)
39.594

>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>> sum(ls)
39.61

All I did was to take the first number, 7.23, and move it to the last position 
in the list.  (So we have a violation of the commutativity of
addition.)

Let me try to reduce the example.  It's not so easy.  Although I could display 
the violation of commutativity by moving just a single number in the list, I 
also see that 7.23 commutes with every other number in the list.

(*) My request

I would like to just get some clarity.  I guess I need to translate all these 
numbers into base 2 and perform the addition myself to see the situation coming 
up?
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: on floating-point numbers

[Schachner, Joseph]

What's really going on is that you are printing out more digits than you are 
entitled to.  39.61 :   16 decimal digits.   4e16 should require 55 
binary bits (in the mantissa) to represent, at least as I calculate it.

Double precision floating point has 52 bits in the mantissa, plus one assumed 
due to normalization.  So 53 bits.

The actual minor difference in sums that you see is because when you put the 
largest value 1st it makes a difference in the last few bits of the mantissa.

I recommend that you print out double precision values to at most 14 digits.  
Then you will never see this kind of issue.  If you don't like that suggestion, 
you can create your own floating point representation using a Python integer as 
the mantissa, so it can grow as large as you have memory to represent the 
value; and a sign and an exponent.  It would be slow, but it could have much 
more accuracy (if implemented to preserve accuracy).

By the way, this is why banks and other financial institutions use BCD (binary 
coded decimal).   They cannot tolerate sums that have fraction of a cent errors.

I should also point out another float issue: subtractive cancellation.   Try 
1e14 + 0.1  - 1e14. The result clearly should be 0.1, but it won't be.  
That's because 0.1 cannot be accurately represented in binary, and it was only 
represented in the bottom few bits.  I just tried it:   I got 0.09375   
This is not a Python issue.  This is a well known issue when using binary 
floating point.   So, when you sum a large array of data, to avoid these 
issues, you could either
1) sort the data smallest to largest ... may be helpful, but maybe not.
2) Create multiple sums of a few of the values.   Next layer: Sum a few of the 
sums.Top layer: Sum the sum of sums to get the final sum.  This is much 
more likely to work accurately than adding up all the values in one summation 
except the last, and then adding the last (which could be a relatively small 
value).  

--- Joseph S.







Teledyne Confidential; Commercially Sensitive Business Data

-Original Message-
From: Hope Rouselle  
Sent: Thursday, September 2, 2021 9:51 AM
To: python-list@python.org
Subject: on floating-point numbers

Just sharing a case of floating-point numbers.  Nothing needed to be solved or 
to be figured out.  Just bringing up conversation.

(*) An introduction to me

I don't understand floating-point numbers from the inside out, but I do know 
how to work with base 2 and scientific notation.  So the idea of expressing a 
number as 

  mantissa * base^{power}

is not foreign to me. (If that helps you to perhaps instruct me on what's going 
on here.)

(*) A presentation of the behavior

>>> import sys
>>> sys.version
'3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit 
(AMD64)]'

>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>> sum(ls)
39.594

>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>> sum(ls)
39.61

All I did was to take the first number, 7.23, and move it to the last position 
in the list.  (So we have a violation of the commutativity of
addition.)

Let me try to reduce the example.  It's not so easy.  Although I could display 
the violation of commutativity by moving just a single number in the list, I 
also see that 7.23 commutes with every other number in the list.

(*) My request

I would like to just get some clarity.  I guess I need to translate all these 
numbers into base 2 and perform the addition myself to see the situation coming 
up?
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[Brandt Bucher]

Brandt Bucher  added the comment:

I'm compiling Clang now to try to reproduce using a UBSan build (I'm on Ubuntu, 
though).

I'm not entirely familiar with how these sanitizer builds work... could the 
implication be that we're hitting undefined behavior at some point? Or is it 
just a red herring?

Note also that the "set([float('nan'), b'a', b'b', b'c', 'x', 'y', 'z'])" and 
"frozenset([float('nan'), b'a', b'b', b'c', 'x', 'y', 'z'])" tests seem to be 
working just fine... meaning their ordering on this buildbot is different under 
PYTHONHASHSEEDs 0 and 1 (as expected). It may still be a 
platform-or-configuration-dependent ordering, though.

Raymond: off the top of your head, are there any obvious reasons this could be 
happening?

--
assignee:  -> brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[Dong-hee Na]

Dong-hee Na  added the comment:

I like the idea of Py_ALWAYS_INLINE personally if we have to do that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: on floating-point numbers

[MRAB]

On 2021-09-03 16:13, Chris Angelico wrote:

On Sat, Sep 4, 2021 at 12:08 AM o1bigtenor  wrote:

Hmmm - - - ZI would suggest that you haven't looked into
taxation yet!
In taxation you get a rational number that MUST be multiplied by
the amount in currency.


(You can, of course, multiply a currency amount by any scalar. Just
not by another currency amount.)


The error rate here is stupendous.
Some organizations track each transaction with its taxes rounded.
Then some track using  use untaxed and then calculate the taxes
on the whole (when you have 2 or 3 or 4 (dunno about more but
who knows there are some seriously tax loving jurisdictions out there))
the differences between adding amounts and then calculating taxes
and calculating taxes on each amount and then adding all items
together can have some 'interesting' differences.

So financial data MUST be able to handle rational numbers.
(I have been bit by the differences enumerated in the previous!)


The worst problem is knowing WHEN to round. Sometimes you have to do
intermediate rounding in order to make something agree with something
else :(

But if you need finer resolution than the cent, I would still
recommend trying to use fixed-point arithmetic. The trouble is
figuring out exactly how much precision you need. Often, 1c precision
is actually sufficient.

At some point, some finance/legal person has to specify how any 
fractional currency should be handled.

--
https://mail.python.org/mailman/listinfo/python-list

[issue43950] Include column offsets for bytecode instructions

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:


New changeset 85ea2d6165dec0cffa6302eb6dc40406eae1edf5 by Batuhan Taskaya in 
branch 'main':
bpo-43950: support positions for dis.Instructions created through dis.Bytecode 
(GH-28142)
https://github.com/python/cpython/commit/85ea2d6165dec0cffa6302eb6dc40406eae1edf5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: on floating-point numbers

[Chris Angelico]

On Sat, Sep 4, 2021 at 12:08 AM o1bigtenor  wrote:
> Hmmm - - - ZI would suggest that you haven't looked into
> taxation yet!
> In taxation you get a rational number that MUST be multiplied by
> the amount in currency.

(You can, of course, multiply a currency amount by any scalar. Just
not by another currency amount.)

> The error rate here is stupendous.
> Some organizations track each transaction with its taxes rounded.
> Then some track using  use untaxed and then calculate the taxes
> on the whole (when you have 2 or 3 or 4 (dunno about more but
> who knows there are some seriously tax loving jurisdictions out there))
> the differences between adding amounts and then calculating taxes
> and calculating taxes on each amount and then adding all items
> together can have some 'interesting' differences.
>
> So financial data MUST be able to handle rational numbers.
> (I have been bit by the differences enumerated in the previous!)

The worst problem is knowing WHEN to round. Sometimes you have to do
intermediate rounding in order to make something agree with something
else :(

But if you need finer resolution than the cent, I would still
recommend trying to use fixed-point arithmetic. The trouble is
figuring out exactly how much precision you need. Often, 1c precision
is actually sufficient.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43950] Include column offsets for bytecode instructions

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +26581
pull_request: https://github.com/python/cpython/pull/28142

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +26580
pull_request: https://github.com/python/cpython/pull/28141

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7974c30b9fd84fa56ea1515ed2c08b38edf1a383 by Victor Stinner in 
branch 'main':
bpo-45094: Add Py_NO_INLINE macro (GH-28140)
https://github.com/python/cpython/commit/7974c30b9fd84fa56ea1515ed2c08b38edf1a383


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7974c30b9fd84fa56ea1515ed2c08b38edf1a383 by Victor Stinner in 
branch 'main':
bpo-45094: Add Py_NO_INLINE macro (GH-28140)
https://github.com/python/cpython/commit/7974c30b9fd84fa56ea1515ed2c08b38edf1a383


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7974c30b9fd84fa56ea1515ed2c08b38edf1a383 by Victor Stinner in 
branch 'main':
bpo-45094: Add Py_NO_INLINE macro (GH-28140)
https://github.com/python/cpython/commit/7974c30b9fd84fa56ea1515ed2c08b38edf1a383


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7974c30b9fd84fa56ea1515ed2c08b38edf1a383 by Victor Stinner in 
branch 'main':
bpo-45094: Add Py_NO_INLINE macro (GH-28140)
https://github.com/python/cpython/commit/7974c30b9fd84fa56ea1515ed2c08b38edf1a383


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: on floating-point numbers

[o1bigtenor]

On Thu, Sep 2, 2021 at 2:27 PM Chris Angelico  wrote:

> On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle  wrote:
> >
> > Hope Rouselle  writes:
> >
> > > Just sharing a case of floating-point numbers.  Nothing needed to be
> > > solved or to be figured out.  Just bringing up conversation.
> > >
> > > (*) An introduction to me
> > >
> > > I don't understand floating-point numbers from the inside out, but I do
> > > know how to work with base 2 and scientific notation.  So the idea of
> > > expressing a number as
> > >
> > >   mantissa * base^{power}
> > >
> > > is not foreign to me. (If that helps you to perhaps instruct me on
> > > what's going on here.)
> > >
> > > (*) A presentation of the behavior
> > >
> >  import sys
> >  sys.version
> > > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
> > > bit (AMD64)]'
> > >
> >  ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
> >  sum(ls)
> > > 39.594
> > >
> >  ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
> >  sum(ls)
> > > 39.61
> > >
> > > All I did was to take the first number, 7.23, and move it to the last
> > > position in the list.  (So we have a violation of the commutativity of
> > > addition.)
> >
> > Suppose these numbers are prices in dollar, never going beyond cents.
> > Would it be safe to multiply each one of them by 100 and therefore work
> > with cents only?  For instance
>
> Yes and no. It absolutely *is* safe to always work with cents, but to
> do that, you have to be consistent: ALWAYS work with cents, never with
> floating point dollars.
>
> (Or whatever other unit you choose to use. Most currencies have a
> smallest-normally-used-unit, with other currency units (where present)
> being whole number multiples of that minimal unit. Only in forex do
> you need to concern yourself with fractional cents or fractional yen.)
>
> But multiplying a set of floats by 100 won't necessarily solve your
> problem; you may have already fallen victim to the flaw of assuming
> that the numbers are represented accurately.
>
> > --8<---cut here---start->8---
> > >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
> > >>> sum(map(lambda x: int(x*100), ls)) / 100
> > 39.6
> >
> > >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
> > >>> sum(map(lambda x: int(x*100), ls)) / 100
> > 39.6
> > --8<---cut here---end--->8---
> >
> > Or multiplication by 100 isn't quite ``safe'' to do with floating-point
> > numbers either?  (It worked in this case.)
>
> You're multiplying and then truncating, which risks a round-down
> error. Try adding a half onto them first:
>
> int(x * 100 + 0.5)
>
> But that's still not a perfect guarantee. Far safer would be to
> consider monetary values to be a different type of value, not just a
> raw number. For instance, the value $7.23 could be stored internally
> as the integer 723, but you also know that it's a value in USD, not a
> simple scalar. It makes perfect sense to add USD+USD, it makes perfect
> sense to multiply USD*scalar, but it doesn't make sense to multiply
> USD*USD.
>
> > I suppose that if I multiply it by a power of two, that would be an
> > operation that I can be sure will not bring about any precision loss
> > with floating-point numbers.  Do you agree?
>
> Assuming you're nowhere near 2**53, yes, that would be safe. But so
> would multiplying by a power of five. The problem isn't precision loss
> from the multiplication - the problem is that your input numbers
> aren't what you think they are. That number 7.23, for instance, is
> really
>
> >>> 7.23.as_integer_ratio()
> (2035064081618043, 281474976710656)
>
> ... the rational number 2035064081618043 / 281474976710656, which is
> very close to 7.23, but not exactly so. (The numerator would have to
> be ...8042.88 to be exactly correct.) There is nothing you can do at
> this point to regain the precision, although a bit of multiplication
> and rounding can cheat it and make it appear as if you did.
>
> Floating point is a very useful approximation to real numbers, but
> real numbers aren't the best way to represent financial data. Integers
> are.
>
>
Hmmm - - - ZI would suggest that you haven't looked into
taxation yet!
In taxation you get a rational number that MUST be multiplied by
the amount in currency.
The error rate here is stupendous.
Some organizations track each transaction with its taxes rounded.
Then some track using  use untaxed and then calculate the taxes
on the whole (when you have 2 or 3 or 4 (dunno about more but
who knows there are some seriously tax loving jurisdictions out there))
the differences between adding amounts and then calculating taxes
and calculating taxes on each amount and then adding all items
together can have some 'interesting' differences.

So financial data MUST be able to handle rational numbers.
(I have been bit by the differences enumerated in the previous!)

Regards
-- 

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +26579
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28140

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Connecting python to DB2 database

[Chris Angelico]

On Fri, Sep 3, 2021 at 11:37 PM DFS  wrote:
>
> On 9/3/2021 1:47 AM, Chris Angelico wrote:
> > On Fri, Sep 3, 2021 at 3:42 PM DFS  wrote:
> >>
> >> Having a problem with the DB2 connector
> >>
> >> test.py
> >> 
> >> import ibm_db_dbi
> >> connectstring =
> >> 'DATABASE=xxx;HOSTNAME=localhost;PORT=5;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;'
> >> conn = ibm_db_dbi.connect(connectstring,'','')
> >>
> >> curr  = conn.cursor
> >> print(curr)
> >
> > According to PEP 249, what you want is conn.cursor() not conn.cursor.
> >
> > I'm a bit surprised as to the repr of that function though, which
> > seems to be this line from your output:
> >
> > 
> >
> > I'd have expected it to say something like "method cursor of
> > Connection object", which would have been an immediate clue as to what
> > needs to be done. Not sure why the repr is so confusing, and that
> > might be something to report upstream.
> >
> > ChrisA
>
>
> Thanks.  I must've done it right, using conn.cursor(), 500x.
> Bleary-eyed from staring at code too long I guess.

Cool cool! Glad that's working.

> Now can you get DB2 to accept ; as a SQL statement terminator like the
> rest of the world?   They call it "An unexpected token"...
>

Hmm, I don't know that the execute() method guarantees to allow
semicolons. Some implementations will strip a trailing semi, but they
usually won't allow interior ones, because that's a good way to worsen
SQL injection vulnerabilities. It's entirely possible - and within the
PEP 249 spec, I believe - for semicolons to be simply rejected.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[Brandt Bucher]

Brandt Bucher  added the comment:

Thanks for finding this, Victor.

That failure is surprising to me. Is it really possible for the order of the 
elements in a set to vary based on platform or build configuration (even with a 
fixed PYTHONHASHSEED at runtime)?

Really, this looks like it’s only a bug in the test’s (read “my”) assumptions, 
not really in marshal itself. I’m glad I added this little sanity check, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Connecting python to DB2 database

[DFS]

On 9/3/2021 1:47 AM, Chris Angelico wrote:

On Fri, Sep 3, 2021 at 3:42 PM DFS  wrote:


Having a problem with the DB2 connector

test.py

import ibm_db_dbi
connectstring =
'DATABASE=xxx;HOSTNAME=localhost;PORT=5;PROTOCOL=TCPIP;UID=xxx;PWD=xxx;'
conn = ibm_db_dbi.connect(connectstring,'','')

curr  = conn.cursor
print(curr)


According to PEP 249, what you want is conn.cursor() not conn.cursor.

I'm a bit surprised as to the repr of that function though, which
seems to be this line from your output:



I'd have expected it to say something like "method cursor of
Connection object", which would have been an immediate clue as to what
needs to be done. Not sure why the repr is so confusing, and that
might be something to report upstream.

ChrisA



Thanks.  I must've done it right, using conn.cursor(), 500x. 
Bleary-eyed from staring at code too long I guess.


Now can you get DB2 to accept ; as a SQL statement terminator like the 
rest of the world?   They call it "An unexpected token"...


--
https://mail.python.org/mailman/listinfo/python-list

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[Ken Jin]

Change by Ken Jin :


--
nosy: +kj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44348] test_exceptions.ExceptionTests.test_recursion_in_except_handler stack overflow on Windows debug builds

[STINNER Victor]

STINNER Victor  added the comment:

See bpo-45094: "Consider using __forceinline and __attribute__((always_inline)) 
on static inline functions (Py_INCREF, Py_TYPE) for debug builds".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35059] Convert Py_INCREF() and PyObject_INIT() to inlined functions

[STINNER Victor]

STINNER Victor  added the comment:

See bpo-45094: "Consider using __forceinline and __attribute__((always_inline)) 
on static inline functions (Py_INCREF, Py_TYPE) for debug builds".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +corona10, erlendaasland, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds

[STINNER Victor]

Change by STINNER Victor :


--
title: Consider using __forceinline and __attribute__((always_inline)) for 
debug builds -> Consider using __forceinline and __attribute__((always_inline)) 
on static inline functions (Py_INCREF, Py_TYPE) for debug builds

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45094] Consider using __forceinline and __attribute__((always_inline)) for debug builds

[STINNER Victor]

New submission from STINNER Victor :

I converted many C macros to static inline functions to reduce the risk of 
programming bugs (macros pitfalls), limit variable scopes to the function, have 
a more readable function, and benefit of the function name even when it's 
inlined in debuggers and profilers.

When Py_INCREF() macro was converted to a static inline function, using 
__attribute__((always_inline)) was considered, but the idea was rejected. See 
bpo-35059.

I'm now trying to convert the Py_TYPE() macro to a static inline function. The 
problem is that by default, MSC disables inlining and test_exceptions does 
crash with a stack overflow, since my PR 28128 increases the usage of the stack 
memory: see bpo-44348.

For the specific case of CPython built by MSC, we can increase the stack size, 
or change compiler optimizations to enable inlining. But the problem is wider 
than just CPython built by MSC in debug mode. Third party C extensions built by 
distutils may get the same issue. Building CPython on other platforms on debug 
mode with all compiler optimizations disabled (ex: gcc -O0) can also have the 
same issue.

I propose to reconsider the usage __forceinline (MSC) and 
__attribute__((always_inline)) (GCC, clang) on the most important static inline 
functions, like Py_INCREF() and Py_TYPE(), to avoid this issue.

--
components: Interpreter Core
messages: 400990
nosy: vstinner
priority: normal
severity: normal
status: open
title: Consider using __forceinline and __attribute__((always_inline)) for 
debug builds
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: on floating-point numbers

[Oscar Benjamin]

On Fri, 3 Sept 2021 at 13:48, Chris Angelico  wrote:
>
> On Fri, Sep 3, 2021 at 10:42 PM jak  wrote:
> >
> > Il 03/09/2021 09:07, Julio Di Egidio ha scritto:
> > > On Friday, 3 September 2021 at 01:22:28 UTC+2, Chris Angelico wrote:
> > >> On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber  
> > >> wrote:
> > >>> On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico 
> > >>> declaimed the following:
> > >>>
> >  The naive summation algorithm used by sum() is compatible with a
> >  variety of different data types - even lists, although it's documented
> >  as being intended for numbers - but if you know for sure that you're
> >  working with floats, there's a more accurate algorithm available to
> >  you.
> > 
> > >>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77])
> >  39.6
> > >>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23])
> >  39.6
> > 
> >  It seeks to minimize loss to repeated rounding and is, I believe,
> >  independent of data order.
> > >>>
> > >>> Most likely it sorts the data so the smallest values get summed first,
> > >>> and works its way up to the larger values. That way it minimizes the 
> > >>> losses
> > >>> that occur when denormalizing a value (to set the exponent equal to 
> > >>> that of
> > >>> the next larger value).
> > >>>
> > >> I'm not sure, but that sounds familiar. It doesn't really matter
> > >> though - the docs just say that it is an "accurate floating point
> > >> sum", so the precise algorithm is an implementation detail.
> > >
> > > The docs are quite misleading there, it is not accurate without further 
> > > qualifications.
> > >
> > > 
> > > 
> > >
> >
> > https://en.wikipedia.org/wiki/IEEE_754
>
> I believe the definition of "accurate" here is that, if you take all
> of the real numbers represented by those floats, add them all together
> with mathematical accuracy, and then take the nearest representable
> float, that will be the exact value that fsum will return. In other
> words, its accuracy is exactly as good as the final result can be.

It's as good as it can be if the result must fit into a single float.
Actually the algorithm itself maintains an exact result for the sum
internally using a list of floats whose exact sum is the same as that
of the input list. In essence it compresses a large list of floats to
a small list of say 2 or 3 floats while preserving the exact value of
the sum.

Unfortunately fsum does not give any way to access the internal exact
list so using fsum repeatedly suffers the same problems as plain float
arithmetic e.g.:
>>> x = 10**20
>>> fsum([fsum([1, x]), -x])
0.0

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[Chris Angelico]

On Fri, Sep 3, 2021 at 10:42 PM jak  wrote:
>
> Il 03/09/2021 09:07, Julio Di Egidio ha scritto:
> > On Friday, 3 September 2021 at 01:22:28 UTC+2, Chris Angelico wrote:
> >> On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber  
> >> wrote:
> >>> On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico 
> >>> declaimed the following:
> >>>
>  The naive summation algorithm used by sum() is compatible with a
>  variety of different data types - even lists, although it's documented
>  as being intended for numbers - but if you know for sure that you're
>  working with floats, there's a more accurate algorithm available to
>  you.
> 
> >>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77])
>  39.6
> >>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23])
>  39.6
> 
>  It seeks to minimize loss to repeated rounding and is, I believe,
>  independent of data order.
> >>>
> >>> Most likely it sorts the data so the smallest values get summed first,
> >>> and works its way up to the larger values. That way it minimizes the 
> >>> losses
> >>> that occur when denormalizing a value (to set the exponent equal to that 
> >>> of
> >>> the next larger value).
> >>>
> >> I'm not sure, but that sounds familiar. It doesn't really matter
> >> though - the docs just say that it is an "accurate floating point
> >> sum", so the precise algorithm is an implementation detail.
> >
> > The docs are quite misleading there, it is not accurate without further 
> > qualifications.
> >
> > 
> > 
> >
>
> https://en.wikipedia.org/wiki/IEEE_754

I believe the definition of "accurate" here is that, if you take all
of the real numbers represented by those floats, add them all together
with mathematical accuracy, and then take the nearest representable
float, that will be the exact value that fsum will return. In other
words, its accuracy is exactly as good as the final result can be.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[jak]

Il 03/09/2021 09:07, Julio Di Egidio ha scritto:

On Friday, 3 September 2021 at 01:22:28 UTC+2, Chris Angelico wrote:

On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber  wrote:

On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico 
declaimed the following:


The naive summation algorithm used by sum() is compatible with a
variety of different data types - even lists, although it's documented
as being intended for numbers - but if you know for sure that you're
working with floats, there's a more accurate algorithm available to
you.


math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77])

39.6

math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23])

39.6

It seeks to minimize loss to repeated rounding and is, I believe,
independent of data order.


Most likely it sorts the data so the smallest values get summed first,
and works its way up to the larger values. That way it minimizes the losses
that occur when denormalizing a value (to set the exponent equal to that of
the next larger value).


I'm not sure, but that sounds familiar. It doesn't really matter
though - the docs just say that it is an "accurate floating point
sum", so the precise algorithm is an implementation detail.


The docs are quite misleading there, it is not accurate without further 
qualifications.




That said, fucking pathetic, when Dunning-Kruger is a compliment...

*Plonk*

Julio



https://en.wikipedia.org/wiki/IEEE_754
--
https://mail.python.org/mailman/listinfo/python-list

Re: The sqlite3 timestamp conversion between unixepoch and localtime

[Bob Martin]

On 2 Sep 2021 at 20:25:27, Alan Gauld  wrote:
> On 02/09/2021 20:11, MRAB wrote:
>
>>> In one of them (I can't recall which is which) they change on the 4th
>>> weekend of October/March in the other they change on the last weekend.
>>>
>>>
>> In the EU (and UK) it's the last Sunday in March/October.
>>
>> In the US it's second Sunday in March and the first Sunday in November.
>>
>> I know which one I find easier to remember!
>
> Interesting. I remember it as closer than that. The bugs we found were
> due to differences in the DST settings of the BIOS in the PCs. (They
> were deliberately all sourced from DELL but the EU PCs had a slightly
> different BIOS).
>
> The differences you cite should have thrown up issues every year.
> I must see if I can find my old log books...
>

ISTR that the USA changes were the same as the EU until a few years ago.

I remember thinking at the time it changed "why would they do that?"

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[Christian Gollwitzer]

Am 02.09.21 um 21:02 schrieb Julio Di Egidio:

On Thursday, 2 September 2021 at 20:43:36 UTC+2, Chris Angelico wrote:

On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle  wrote:



All I did was to take the first number, 7.23, and move it to the last
position in the list. (So we have a violation of the commutativity of
addition.)


It's not about the commutativity of any particular pair of operands -
that's always guaranteed.


Nope, that is rather *not* guaranteed, as I have quite explained up thread.



No, you haven't explained that. You linked to the famous Goldberg paper. 
Where in the paper does it say that operations on floats are not 
commutative?


I'd be surprised because it is generally wrong.
Unless you have special numbers like NaN or signed zeros etc., a+b=b+a 
and a*b=b*a holds also for floats.


Christiah
--
https://mail.python.org/mailman/listinfo/python-list

[issue41031] Inconsistency in C and python traceback printers

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41031] Inconsistency in C and python traceback printers

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +patch
nosy: +iritkatriel
nosy_count: 1.0 -> 2.0
pull_requests: +26578
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28139

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[STINNER Victor]

STINNER Victor  added the comment:

The test failed at:

def test_deterministic_sets(self):
# bpo-37596: To support reproducible builds, sets and frozensets need to
# have their elements serialized in a consistent order (even when they
# have been scrambled by hash randomization):
for kind in ("set", "frozenset"):
for elements in (
"float('nan'), b'a', b'b', b'c', 'x', 'y', 'z'",
# Also test for bad interactions with backreferencing:
"('string', 1), ('string', 2), ('string', 3)",
):
s = f"{kind}([{elements}])"
with self.subTest(s):
# First, make sure that our test case still has different
# orders under hash seeds 0 and 1. If this check fails, we
# need to update this test with different elements:
args = ["-c", f"print({s})"]
_, repr_0, _ = assert_python_ok(*args, PYTHONHASHSEED="0")
_, repr_1, _ = assert_python_ok(*args, PYTHONHASHSEED="1")
self.assertNotEqual(repr_0, repr_1)  # <=== HERE
(...)

It checks that the representation of a set is different for two different 
PYTHONHASHSEED values (0 and 1). On my Fedora 34, I confirm that they are 
different:

PYTHONHASHSEED=0:

vstinner@apu$ PYTHONHASHSEED=0 ./python -c "print(set([('string', 1), 
('string', 2), ('string', 3)]))"
{('string', 1), ('string', 2), ('string', 3)}
vstinner@apu$ PYTHONHASHSEED=0 ./python -c "print(set([('string', 1), 
('string', 2), ('string', 3)]))"
{('string', 1), ('string', 2), ('string', 3)}
vstinner@apu$ PYTHONHASHSEED=0 ./python -c "print(set([('string', 1), 
('string', 2), ('string', 3)]))"
{('string', 1), ('string', 2), ('string', 3)}

versus PYTHONHASHSEED=1:

vstinner@apu$ PYTHONHASHSEED=1 ./python -c "print(set([('string', 1), 
('string', 2), ('string', 3)]))"
{('string', 3), ('string', 1), ('string', 2)}
vstinner@apu$ PYTHONHASHSEED=1 ./python -c "print(set([('string', 1), 
('string', 2), ('string', 3)]))"
{('string', 3), ('string', 1), ('string', 2)}
vstinner@apu$ PYTHONHASHSEED=1 ./python -c "print(set([('string', 1), 
('string', 2), ('string', 3)]))"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[STINNER Victor]

STINNER Victor  added the comment:

I reopen the issue.

test_marshal failed on AMD64 Arch Linux Usan 3.x:
https://buildbot.python.org/all/#/builders/719/builds/108

==
FAIL: test_deterministic_sets (test.test_marshal.BugsTestCase) [set([('string', 
1), ('string', 2), ('string', 3)])]
--
Traceback (most recent call last):
  File 
"/buildbot/buildarea/3.x.pablogsal-arch-x86_64.clang-ubsan/build/Lib/test/test_marshal.py",
 line 365, in test_deterministic_sets
self.assertNotEqual(repr_0, repr_1)
^^^
AssertionError: b"{('string', 1), ('string', 2), ('string', 3)}\n" == 
b"{('string', 1), ('string', 2), ('string', 3)}\n"

==
FAIL: test_deterministic_sets (test.test_marshal.BugsTestCase) 
[frozenset([('string', 1), ('string', 2), ('string', 3)])]
--
Traceback (most recent call last):
  File 
"/buildbot/buildarea/3.x.pablogsal-arch-x86_64.clang-ubsan/build/Lib/test/test_marshal.py",
 line 365, in test_deterministic_sets
self.assertNotEqual(repr_0, repr_1)
^^^
AssertionError: b"frozenset({('string', 1), ('string', 2), ('string', 3)})\n" 
== b"frozenset({('string', 1), ('string', 2), ('string', 3)})\n"

--
nosy: +vstinner
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Inada Naoki]

Change by Inada Naoki :


--
keywords: +patch
pull_requests: +26577
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28138

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42238] Deprecate suspicious.py?

[Julien Palard]

Change by Julien Palard :


--
pull_requests: +26576
pull_request: https://github.com/python/cpython/pull/28137

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24665] CJK support for textwrap

[Tobias Bengfort]

Change by Tobias Bengfort :


--
nosy: +xi2
nosy_count: 10.0 -> 11.0
pull_requests: +26575
pull_request: https://github.com/python/cpython/pull/28136

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12499] textwrap.wrap: add control for fonts with different character widths

[Tobias Bengfort]

Change by Tobias Bengfort :


--
pull_requests: +26574
pull_request: https://github.com/python/cpython/pull/28136

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12499] textwrap.wrap: add control for fonts with different character widths

[Tobias Bengfort]

Change by Tobias Bengfort :


--
nosy: +xi2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45077] multiprocessing.Pool(64) crashes on Windows

[Eryk Sun]

Eryk Sun  added the comment:

See bpo-26903 for a similar problem in concurrent.futures.ProcessPoolExecutor. 
It was resolved by adding a limit constant, _MAX_WINDOWS_WORKERS == 61. 

WaitForMultipleObjects() can wait on up to 64 object handles, but in this case 
3 slots are already taken. The pool wait includes two events for its output and 
change-notifier queues (named pipes), plus the _winapi module always reserves a 
slot for the SIGINT event, even though this event is only used by waits on the 
main thread.

To avoid the need to limit the pool size, connection._exhaustive_wait() could 
be modified to combine simultaneous waits on up to 63 threads, for which each 
thread exhaustively populates a list of up to 64 signaled objects. I wouldn't 
want to modify _winapi.WaitForMultipleObjects, but the exhaustive wait should 
still be implemented in C, probably in the _multiprocessing extension module. A 
benefit of implementing _exhaustive_wait() in C is lightweight thread creation, 
directly with CreateThread() and a relatively small stack commit size.

--
components: +Library (Lib)
nosy: +eryksun
versions: +Python 3.10, Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com