[issue33328] pdb error when stepping through generator

[Rick Teachey]

Rick Teachey  added the comment:

I am on Anaconda 4.5.1 on Windows 10 (Python 3.6.5). I have confirmed that I DO 
get the error on another machine with the same version installed, so the lack 
of an error seems specific to my current machine. I do not know what could be 
causing this; it is very odd.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33328] pdb error when stepping through generator

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you for your report Rick. But I get the same error in 3.6.

An AttributeError is raised if the debugger runs at the interpreter shutdown 
stage (sys.is_finalizing() returns True). sys.path was set to None at the 
beginning of modules cleanup. The debugger traces the _ag() asynchronous 
generator in types.py. You can see some details when run Python with option -v.

$ ./python -v ../demo.py
...
import 'pdb' # <_frozen_importlib_external.SourceFileLoader object at 
0x7f5b45debae8>
# extension module 'readline' loaded from 
'/home/serhiy/py/cpython/build/lib.linux-x86_64-3.8-pydebug/readline.cpython-38dm-x86_64-linux-gnu.so'
# extension module 'readline' executed from 
'/home/serhiy/py/cpython/build/lib.linux-x86_64-3.8-pydebug/readline.cpython-38dm-x86_64-linux-gnu.so'
import 'readline' # <_frozen_importlib_external.ExtensionFileLoader object at 
0x7f5b45d94608>
GeneratorExit

# clear builtins._
# clear sys.path
# clear sys.argv
...
# cleanup[3] wiping re
# destroy _locale
# destroy sre_compile
# destroy copyreg
# cleanup[3] wiping enum
# cleanup[3] wiping types
Exception ignored in: 
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/types.py", line 27, in _ag
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 90, in trace_dispatch
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 128, in dispatch_call
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 250, in break_anywhere
  File "/home/serhiy/py/cpython/Lib/bdb.py", line 49, in canonic
AttributeError: 'NoneType' object has no attribute 'abspath'

Adding "del _ag" in types.py fixes this issue. But there are open questions.

Is it good that the debugger is enabled at the shutdown stage?

Why the code of the never used asynchronous generator is executed at all?

--
nosy: +pitrou, serhiy.storchaka, yselivanov
versions: +Python 3.6, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33328] pdb error when stepping through generator

[Rick Teachey]

Change by Rick Teachey :


--
components: +Library (Lib)
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33328] pdb error when stepping through generator

[Rick Teachey]

Rick Teachey  added the comment:

The interactive session result is below:

GeneratorExit

Exception ignored in: 
Traceback (most recent call last):
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\types.py", 
line 27, in _ag
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 
90, in trace_dispatch
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 
128, in dispatch_call
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 
250, in break_anywhere
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\bdb.py", line 
49, in canonic
AttributeError: 'NoneType' object has no attribute 'abspath'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33328] pdb error when stepping through generator

[Rick Teachey]

New submission from Rick Teachey :

Hello: 

The attached python file runs a pdb interactive session for a generator. It 
produces an error in Python 3.7 Beta 3, but no error in 3.6.

I searched for this issue and there do seem to be things that are related to it 
already, but I haven't investigated the cause of this issue and am not 
acquainted with the details of the pdb module, so I was unable to determine 
whether previous reports discuss this same problem.

--
files: demo.py
messages: 315586
nosy: Ricyteach
priority: normal
severity: normal
status: open
title: pdb error when stepping through generator
versions: Python 3.7
Added file: https://bugs.python.org/file47545/demo.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33277] Deprecate __loader__, __package__, __file__, and __cached__ on modules

[pmpp]

Change by pmpp :


--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33277] Deprecate __loader__, __package__, __file__, and __cached__ on modules

[Antoine Pitrou]

Antoine Pitrou  added the comment:

This sounds like gratuitous breakage to me.  __file__ in particular is used 
quite often.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31463] test_multiprocessing_fork hangs test_subprocess

[Antoine Pitrou]

Antoine Pitrou  added the comment:

There's a test that deliberately kills the semaphore tracker, so that warning 
should be expected (I don't see it myself, but that might be because of 
differing warning settings).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31463] test_multiprocessing_fork hangs test_subprocess

[Benjamin Peterson]

Benjamin Peterson  added the comment:

It doesn't hang for me anymore, but does print this warning:

0:00:00 load avg: 1.59 [1/2] test_multiprocessing_fork
Lib/multiprocessing/semaphore_tracker.py:55: UserWarning: semaphore_tracker: 
process died unexpectedly, relaunching.  Some semaphores might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32911] Doc strings no longer stored in body of AST

[Łukasz Langa]

Łukasz Langa  added the comment:

Oh, and Francis, you might actually want to use lib2to3.pytree instead, which 
will provide you with a Concrete Syntax Tree ("lossless AST").  That tree will 
keep information like whitespace, comments, and other semantically 
insignificant tokens like commas and parentheses.

This tree will never modify your input whereas in the AST we might be doing 
more modifications in the future (like cutting `if` branches that we statically 
know are always False, and so on).

Granted, the lib2to3 tree is a bit awkward to query since the concrete tree 
exposes the raw Python grammar and not the simplified AST grammar.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32911] Doc strings no longer stored in body of AST

[Łukasz Langa]

Łukasz Langa  added the comment:

Inadasan, what Francis is referring to is that people often use `ast.parse()` 
to parse code fragments, not entire modules.

For example:

  >>> some_code = '"just a string"'
  >>> some_code_ast = ast.parse(some_code)

The way `ast.parse()` works, it always considers the input to be a module:

  >>> some_code_ast
  <_ast.Module object at 0x109537a18>

Before Python 3.7, you could parse out a string from such "module":

  >>> some_code.body
  [<_ast.Expr object at 0x1079aaa20>]
  >>> some_code.body[0].value
  <_ast.Str object at 0x107a45198>

However, starting with Python 3.7, this module is empty:

  >>> some_code_ast.body
  []
  >>> some_code_ast.docstring
  'just a string'

`ast.literal_eval()` does not have this problem.


And now: where is the bug?  I think the user code has a bug trying to parse a 
code fragment with `mode="exec"`.  Instead of using the default `mode="exec"`, 
it seems to me `mode="single"` fits the bill better.  In some cases the 
programmer might prefer `mode="eval"` if they don't want to allow statements at 
all.

Inadasan, I think what we should do is to amend `ast.parse()` and `compile()` 
docs that mode="exec" treats given code as a module, we should even give the 
docstring handling as an example.

Francis, if your use case requires passing multiple statements/expressions at 
once and a string in the first statement has semantic meaning, please provide 
us with a concrete example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31463] test_multiprocessing_fork hangs test_subprocess

[Antoine Pitrou]

Antoine Pitrou  added the comment:

@Benjamin

I can't reproduce here.  It's true that test_multiprocessing_fork leaves two 
processes being itself, but that doesn't seem to disturb test_subprocess.

@Miro

Could you report the signal number issue separately?  Thank you!

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28320] Hostname validation is False by default in imaplib

[Matej Cepl]

Matej Cepl  added the comment:

See also issue 33327.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33327] Add a method to move messages to IMAPlib

[Matej Cepl]

New submission from Matej Cepl :

I am in the process of writing script working with IMAP first time using Python 
3 for it (unfortunately, most of servers where I run other code is so ancient 
that even Python 2.7 is a stretch), and it is really nice experience so far. 
Many problems which are dealt with on the StackExchange with arcane workarounds 
are now resolved in email or imaplib libraries. Thank you, everybody!

However, it seems to me that few higher level commands could greatly improve 
useability of imaplib library. For example, moving messages (not fault of 
imaplib) is still horrible mess. In the end I had to write this monstrosity 
just to make moving messages working:


Capas = collections.namedtuple('Capas', ['MOVE', 'UIDPLUS'])

def __login(self, host='localhost', username=None, password=None, ssl=None):
self.box = imaplib.IMAP4_SSL(host=host)
ok, data = self.box.login(username, password)
if ok != 'OK':
raise IOError('Cannot login with credentials %s' %
  str((host, username, password,)))

ok, data = box.capability()
capas = data[0].decode()
box.features_present = Capas._make(['MOVE' in capas, 'UIDPLUS' in 
capas])

def move_messages(self, target, messages):
if self.box.features_present.MOVE:
ok, data = self.box.uid('MOVE', '%s %s' % (messages, target))
if ok != 'OK':
raise IOError('Cannot move messages to folder %s' % target)
elif self.box.features_present.UIDPLUS:
ok, data = self.box.uid('COPY', '%s %s' % (messages, target))
if ok != 'OK':
raise IOError('Cannot copy messages to folder %s' % target)
ok, data = self.box.uid('STORE',
r'+FLAGS.SILENT (\DELETED) %s' % messages)
if ok != 'OK':
raise IOError('Cannot delete messages.')
ok, data = self.box.uid('EXPUNGE', messages)
if ok != 'OK':
raise IOError('Cannot expunge messages.')
else:
ok, data = self.box.uid('COPY', '%s %s' % (messages, target))
if ok != 'OK':
raise IOError('Cannot copy messages to folder %s' % target)
ok, data = self.box.uid('STORE',
r'+FLAGS.SILENT (\DELETED) %s' % messages)
if ok != 'OK':
raise IOError('Cannot delete messages.')

It would be nice if some capabilities detection (see issue 18921) was embedded 
into login method, and if some version of move_messages was included in imaplib 
itself.

--
components: Library (Lib)
messages: 315578
nosy: mcepl
priority: normal
severity: normal
status: open
title: Add a method to move messages to IMAPlib
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5430] imaplib: must not replace LF or CR by CRLF in literals

[Matej Cepl]

Matej Cepl  added the comment:

Oh, this is 2.6 bug. This should be closed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33326] Convert collections (cmp_op, hasconst, hasname and others) in opcode module to more optimal type

[Kirill Balunov]

New submission from Kirill Balunov :

The opcode module contains several collections:

`cmp_op`
`hasconst`
`hasname`
`hasjrel`
...

which are only used for `in` checks. At the same time, they are stored as 
`list`s and `cmp_op` as tuple. Both these types are not optimal for 
`__contains__` checks. Maybe it is worth at least to convert them to 
`frozenset` type after they are filled?

--
components: Library (Lib)
messages: 315576
nosy: godaygo
priority: normal
severity: normal
status: open
title: Convert collections (cmp_op, hasconst, hasname and others) in opcode 
module to more optimal type
type: performance
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5430] imaplib: must not replace LF or CR by CRLF in literals

[Matej Cepl]

Matej Cepl  added the comment:

> Module imaplib has pretty sparse test code.

On that note: would anybody be willing to express in form of (simplified) test 
case, what's the problem reported here? I am not sure, I follow.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28320] Hostname validation is False by default in imaplib

[Matej Cepl]

Matej Cepl  added the comment:

I do agree with http://legacy.python.org/dev/peps/pep-0476/#other-protocols:

This PEP only proposes requiring this level of validation for HTTP clients, not 
for other protocols such as SMTP.

This is because while a high percentage of HTTPS servers have correct 
certificates, as a result of the validation performed by browsers, for other 
protocols self-signed or otherwise incorrect certificates are far more common.

With HTTP (and thanks to Let’s Encrypt) the situation seems to be really good, 
and most publicly accessible webserver will hopefully have soon good signed 
certificates, but I am afraid that with other servers (and especially but 
certainly not limited to IMAP servers) there are just too many self-signed 
certificates (or ones signed by suspicious internal CAs) in various internal 
email servers, that changing defaults would do more harm than good, I am 
afraid. Also, arguing about defaults is the way of The Waste of Time, so I will 
try to limit myself just to this one comment on this bug.

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18921] In imaplib, cached capabilities may be out of date after login

[Matej Cepl]

Matej Cepl  added the comment:

Well, and would it be too much to refresh capabilities attribute in the end of 
login method?

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset b9827b762093f2b295934bed755b4de0049b1f15 by Serhiy Storchaka 
(Andrés Delfino) in branch '2.7':
[2.7] bpo-33297: Mention Pillow to work with more image formats. (GH-6505) 
(GH-6562)
https://github.com/python/cpython/commit/b9827b762093f2b295934bed755b4de0049b1f15


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33323] inconsistent stack trace for exceptions thrown in generators passed to min/max

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

In Python 2.7 and 3.6 I got the following traceback for the second case:

Traceback (most recent call last):
  File "issue33323.py", line 21, in 
min(bad(i) for i in range(7)) # unhelpful error message
ValueError: max() arg is an empty sequence

StopIteration in generator expressions didn't treated as an error. It just 
stopped the iteration. The exception is raised by the consumer of generated 
values, min(), because it doesn't work with empty sequences by default.

This behavior caused hard to investigate bugs and was considered harmful. 
Fortunately it will be fixed in Python 3.7. The traceback in 3.7:

Traceback (most recent call last):
  File "issue33323.py", line 21, in 
min(bad(i) for i in range(7)) # unhelpful error message
  File "issue33323.py", line 11, in bad
return n - bad_exception()
  File "issue33323.py", line 5, in bad_exception
return next(iter([]))
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "issue33323.py", line 21, in 
min(bad(i) for i in range(7)) # unhelpful error message
RuntimeError: generator raised StopIteration

You can enable this behavior in earlier versions (3.5+) by adding the future 
import:

from __future__ import generator_stop

See PEP 479 for details.

--
components: +Interpreter Core
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33144] random._randbelow optimization

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee: rhettinger -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33144] random._randbelow optimization

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 6291 didn't work properly with case 1. Rand2 uses getrandbits() since it is 
overridden in the parent despites the fact that random() is defined later.

PR 6563 fixes this. It walks classes in method resolution order and finds the 
first class that defines random() or getrandbits().

PR 6563 also makes tests not using logging for testing purpose.

--
resolution: fixed -> 
stage: resolved -> patch review
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33144] random._randbelow optimization

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +6258

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[Andrés Delfino]

Change by Andrés Delfino :


--
pull_requests: +6257

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33323] inconsistent stack trace for exceptions thrown in generators passed to min/max

[Naris R]

Naris R  added the comment:

This is a little bit contrived but it demonstrates the problem.

```
def good_exception():
raise Exception('something bad happened')

def bad_exception():
return next(iter([]))

def good(n):
return good_exception() + n

def bad(n):
return n - bad_exception()

import traceback

try:
max(good(i) for i in range(4)) # desirable behaviour
except:
traceback.print_exc()

try:
min(bad(i) for i in range(7)) # unhelpful error message
except:
traceback.print_exc()
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33323] inconsistent stack trace for exceptions thrown in generators passed to min/max

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Could you please provide a sample code?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[Andrés Delfino]

Andrés Delfino  added the comment:

Currently cherry picking the commit for 2.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[miss-islington]

miss-islington  added the comment:


New changeset 5c0a8bb1c2c8d77c4b2d5002259d08fe6a277b40 by Miss Islington (bot) 
in branch '3.6':
bpo-33297: Mention Pillow to work with more image formats. (GH-6505)
https://github.com/python/cpython/commit/5c0a8bb1c2c8d77c4b2d5002259d08fe6a277b40


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[miss-islington]

miss-islington  added the comment:


New changeset f9005030fee6af333fada014da3eb0144078779f by Miss Islington (bot) 
in branch '3.7':
bpo-33297: Mention Pillow to work with more image formats. (GH-6505)
https://github.com/python/cpython/commit/f9005030fee6af333fada014da3eb0144078779f


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[miss-islington]

Change by miss-islington :


--
pull_requests: +6256

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset b81ca28b378c8b29204a37f8bd433a3379f53f7d by Serhiy Storchaka 
(Andrés Delfino) in branch 'master':
bpo-33297: Mention Pillow to work with more image formats. (#6505)
https://github.com/python/cpython/commit/b81ca28b378c8b29204a37f8bd433a3379f53f7d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33297] Mention Pillow package on tkinter.rst to work with more image formats

[miss-islington]

Change by miss-islington :


--
pull_requests: +6255

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33325] Optimize sequences of constants in the compiler

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +6254
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33325] Optimize sequences of constants in the compiler

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

The following PR makes three optimizations in the compiler.

1. A sequence of several LOAD_CONSTs is replaced with a single LOAD_CONST 
followed by UNPACK_SEQUENCE.

For example, "{'a': 1, 'b': 2, 'c': 3}" is currently compiled to

  1   0 LOAD_CONST   0 (1)
  2 LOAD_CONST   1 (2)
  4 LOAD_CONST   2 (3)
  6 LOAD_CONST   3 (('a', 'b', 'c'))
  8 BUILD_CONST_KEY_MAP  3
 10 POP_TOP
 12 LOAD_CONST   4 (None)
 14 RETURN_VALUE

With this optimization it will be compiled to:

  1   0 LOAD_CONST   5 ((('a', 'b', 'c'), 3, 2, 1))
  2 UNPACK_SEQUENCE  4
  4 BUILD_CONST_KEY_MAP  3
  6 POP_TOP
  8 LOAD_CONST   4 (None)
 10 RETURN_VALUE

2. Optimized building lists and sets of constants. [1, 2, 3, 4, 5] will be 
compiled to [*(1, 2, 3, 4, 5)], and {1, 2, 3, 4, 5} will be compiled to 
{*frozenset(1, 2, 3, 4, 5)}, where (1, 2, 3, 4, 5) and frozenset(1, 2, 3, 4, 5) 
are just constants.

x = [1, 2, 3, 4, 5]
y = {1, 2, 3, 4, 5}

currently is compiled to

  1   0 LOAD_CONST   0 (1)
  2 LOAD_CONST   1 (2)
  4 LOAD_CONST   2 (3)
  6 LOAD_CONST   3 (4)
  8 LOAD_CONST   4 (5)
 10 BUILD_LIST   5
 12 STORE_NAME   0 (x)

  2  14 LOAD_CONST   0 (1)
 16 LOAD_CONST   1 (2)
 18 LOAD_CONST   2 (3)
 20 LOAD_CONST   3 (4)
 22 LOAD_CONST   4 (5)
 24 BUILD_SET5
 26 STORE_NAME   1 (y)
 28 LOAD_CONST   5 (None)
 30 RETURN_VALUE

With optimization 1 it will be compiled to

  1   0 LOAD_CONST   6 ((5, 4, 3, 2, 1))
  2 UNPACK_SEQUENCE  5
  4 BUILD_LIST   5
  6 STORE_NAME   0 (x)

  2   8 LOAD_CONST   6 ((5, 4, 3, 2, 1))
 10 UNPACK_SEQUENCE  5
 12 BUILD_SET5
 14 STORE_NAME   1 (y)
 16 LOAD_CONST   5 (None)
 18 RETURN_VALUE

And with optimization 2 it will be compiled to

  1   0 LOAD_CONST   0 ((1, 2, 3, 4, 5))
  2 BUILD_LIST_UNPACK1
  4 STORE_NAME   0 (x)

  2   6 LOAD_CONST   1 (frozenset({1, 2, 3, 4, 5}))
  8 BUILD_SET_UNPACK 1
 10 STORE_NAME   1 (y)
 12 LOAD_CONST   2 (None)
 14 RETURN_VALUE

3. Remove unused constants.

After folding tuples of constants created at code generation level, eliminating 
unreachable code, and after the above two optimizations, unused constants are 
left in the co_consts tuple. The third optimization removes them and 
reenumerate constants in the order of occurrence. The above example will be 
compiled to:

  1   0 LOAD_CONST   0 ((1, 2, 3, 4, 5))
  2 BUILD_LIST_UNPACK1
  4 STORE_NAME   0 (x)

  2   6 LOAD_CONST   1 (frozenset({1, 2, 3, 4, 5}))
  8 BUILD_SET_UNPACK 1
 10 STORE_NAME   1 (y)
 12 LOAD_CONST   2 (None)
 14 RETURN_VALUE

See issue28813 for the implementation of this optimization on the level of raw 
bytecode (in peephole.c). 

These optimizations are useful not only for initializing collections of 
constants. Calling function with constant arguments "f(x, a=1, b=2)":

Current:
  1   0 LOAD_NAME0 (f)
  2 LOAD_NAME1 (x)
  4 LOAD_CONST   0 (None)
  6 LOAD_CONST   1 (1)
  8 LOAD_CONST   2 (2)
 10 LOAD_CONST   3 (('a', 'b'))
 12 CALL_FUNCTION_KW 4
 14 POP_TOP
 16 LOAD_CONST   0 (None)
 18 RETURN_VALUE

Optimized:
  1   0 LOAD_NAME0 (f)
  2 LOAD_NAME1 (x)
  4 LOAD_CONST   0 ((('a', 'b'), 2, 1, None))
  6 UNPACK_SEQUENCE  4
  8 CALL_FUNCTION_KW 4
 10 POP_TOP
 12 LOAD_CONST   1 (None)
 14 RETURN_VALUE

This issue depends on issue33318.

--
components: Interpreter Core
messages: 315563
nosy: benjamin.peterson, brett.cannon, inada.naoki, ncoghlan, pitrou, 
rhettinger, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
t

[issue33324] Bug in documentation 3.6: string-methods - str.center

[Alfonso Chavez]

Change by Alfonso Chavez :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33324] Bug in documentation 3.6: string-methods - str.center

[Alfonso Chavez]

New submission from Alfonso Chavez :

Hi. In https://docs.python.org/3/library/stdtypes.html#string-methods says:

str.center(width[, fillchar])

But, I tried in Windows 10 and Ubuntu Server:

>>> var = 'Variable de prueba'
>>> str.center(50, var)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: descriptor 'center' requires a 'str' object but received a 'int'

Then, I tried this:

>>> str.center(var, 50)
'Variable de prueba'

I think the documentation should read:
str.center(fillchar[, width])

Version in Windows: Python 3.6.5
Version in Ubuntu Server: Python 3.5.2

--
assignee: docs@python
components: Documentation
messages: 315562
nosy: Alfonso Chavez, docs@python
priority: normal
severity: normal
status: open
title: Bug in documentation 3.6: string-methods - str.center
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33323] inconsistent stack trace for exceptions thrown in generators passed to min/max

[Naris R]

New submission from Naris R :

if a generator passed to min/max throws an exception, the stack trace is 
normally shown on the line that caused the exception, but if the exception is a 
StopIteration, the trace only shows the line where the max/min function was 
called.

I was writing a minimax and alphabeta search with generator expression and list 
comprehension and accidentally passed an empty iterator to the next function 
while computing states for minimax and it took a very long time to find out 
where the error was actually caused.

--
messages: 315561
nosy: Naris R
priority: normal
severity: normal
status: open
title: inconsistent stack trace for exceptions thrown in generators passed to 
min/max
type: behavior
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24027] IMAP library lacks documentation about expected parameter types

[Matej Cepl]

Matej Cepl  added the comment:

UTF-7 is tackled in https://bugs.python.org/issue5305

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18540] imaplib.IMAP4() ends with "Name or service not known" on Fedora 18

[Matej Cepl]

Matej Cepl  added the comment:

ping?

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26375] Python 2.7.10 and 3.4.4 hang on imaplib.IMAP4_SSL()

[Matej Cepl]

Matej Cepl  added the comment:

Are you able to reproduce it with IMAP4.debug value set to 3 or more? Clean up 
passwords etc. and paste the log here, please. Thank you.

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18280] Documentation is too personalized

[Matej Cepl]

Matej Cepl  added the comment:

What about WONTFIX here? I completely agree with rhettinger: this is a waste of 
time with potential for causing damage.

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5305] imaplib should support international mailbox names

[Matej Cepl]

Change by Matej Cepl :


--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13940] imaplib: Mailbox names are not quoted

[Matej Cepl]

Change by Matej Cepl :


--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32799] returned a result with an error set

[Xiang Zhang]

Xiang Zhang  added the comment:

Our application also runs into this error today and this is an occasional error 
that we didn't encounter before and can't reproduce currently. Although our 
application uses the outdated 2.7.5 but the problem looks similar. Our 
exception traceback points to `PyCell_Get`:

...
RROR nova.api.openstack   File 
"/opt/cloud/services/cascading-nova/venv/lib/python2.7/site-packages/keystonemiddleware/auth_token/_cache.py",
 line 222, in get
RROR nova.api.openstack with self._cache_pool.reserve() as cache:
RROR nova.api.openstack   File "/usr/lib64/python2.7/contextlib.py", line 84, 
in helper
RROR nova.api.openstack return GeneratorContextManager(func(*args, **kwds))
RROR nova.api.openstack SystemError: 
/home/abuild/rpmbuild/BUILD/Python-2.7.5/Objects/cellobject.c:24: bad argument 
to internal function

--
nosy: +xiang.zhang
resolution: works for me -> 
stage: resolved -> 
status: closed -> open
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33289] tkinter askcolor returning floats for r, g, b values instead of ints

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33289] tkinter askcolor returning floats for r, g, b values instead of ints

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Good catch! A color tuple will likely be used in '#%02x%02x%02x' % color, and 
this will fail because %x works only with integers (in general sense). 
Therefore returning a tuple of floats is a bug.

The downside is that we loss some information. Tk supports up to 16 bit per 
color component, and askcolor() keeps only higher 8 of them. This can't be 
changed for backward compatibility, but it may be worth to add an option for 
control the representation of the result. winfo_rgb() returns 16-bit color 
components. This is a separate issue.

--
keywords: +easy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33322] Overridden __getitem__ not called on use of slice syntax when inheriting from tuple

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

In Python 2 things are more complex than in Python 3. You have to define 
__getslice__ for handling the case of literal slices.

This was fixed in Python 3.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33322] Overridden __getitem__ not called on use of slice syntax when inheriting from tuple

[Steven D'Aprano]

Steven D'Aprano  added the comment:

In Python 2, you have to override __getslice__ as well as __getitem__.

Try this instead:


class MyTuple(tuple):
def __getitem__(self, pos):
print "get item {}".format(pos)
return super(MyTuple, self).__getitem__(pos)
def __getslice__(self, *args):
print "get slice {}".format(args)
return super(MyTuple, self).__getslice__(*args)


I don't believe this is a bug. If you try with the above fix, and still believe 
there is a bug, please feel free to re-open the issue.

--
nosy: +steven.daprano
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33318] Move folding tuples of constants into compiler.c from peephole.c

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

For clarifying, this issue doesn't have any relations to the AST optimizer. 
Tuples of constants are already optimized at the AST level, but there are other 
tuples created in code, which don't correspond any AST node. They are worth to 
be optimized too. That is why we need (and have) two codes for folding tuples 
of constants -- at high level (in AST) and at low level (in generated bytecode).

Different kinds of optimizations can be applied at different stages.

1. On AST. Optimizations applied here are: constant folding (evaluating 
expressions with unary and binary operators and subscriptions on constants and 
folding tuples of constants), constant propagation of __debug__, converting "in 
list/set" into "in tuple/frozenset".

2. At code generation stage. Optimizations applied here are: some special cases 
for "if" and "while" with a constant condition, optimizing short-circuit jumps 
in complex boolean expressions, avoiding generating some code (like for "if" 
without "else"), special case for literal dicts with constant keys, merging 
equal constants and names.

3. On generated intermediate representation of bytecode (a graph of sequences 
of instructions). Some unreachable code is eliminated here.

4. On raw bytecode (in peephole.c). Optimizations applied here are: other 
special cases for "if" and "while" with a constant condition, folding tuples of 
constants created at code generation stage, special cases for multitarget 
assignments (1-3 targets), optimizing more short-circuit jumps in complex 
boolean expressions, replacing jumps to unconditional jumps, removing 
unreachable code after RETURN_VALUE.

5. In the code object constructor (it is applied also when unmarshal code). 
Names and identifier-like string constants (including nested in tuples and 
frozensets) are interned here.

PR 6545 moves a particular optimization from stage 4 to stage 3 (to a tiny bit 
higher level). Issue32477 moves other 2 or 3 optimizations from stage 4 to 
stage 3.

It is possible to move this particular optimization even to stage 2, but I'm 
not sure that mixing code generation with optimization would look better. At 
stage 3 it is more isolated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com