[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Ned Deily]

Ned Deily  added the comment:

Thanks, Nick!

--
priority: release blocker -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Nick Coghlan]

Change by Nick Coghlan :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[miss-islington]

miss-islington  added the comment:


New changeset 986eaa86da7ebac4c71a09db88706eea6d89664c by Miss Islington (bot) 
in branch '3.7':
bpo-33185: Improve wording and markup (GH-6477)
https://github.com/python/cpython/commit/986eaa86da7ebac4c71a09db88706eea6d89664c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[miss-islington]

Change by miss-islington :


--
pull_requests: +6178

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Nick Coghlan]

Nick Coghlan  added the comment:


New changeset 1a5c4bdb6ecc6a8b19ff33bde323ab188ed60977 by Nick Coghlan in 
branch 'master':
bpo-33185: Improve wording and markup (GH-6477)
https://github.com/python/cpython/commit/1a5c4bdb6ecc6a8b19ff33bde323ab188ed60977


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Nick Coghlan]

Change by Nick Coghlan :


--
pull_requests: +6177

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[miss-islington]

miss-islington  added the comment:


New changeset d7ffa5820733a528d9ab87ee567738e2d3fd7126 by Miss Islington (bot) 
in branch '3.7':
bpo-33185: Fix regression in pydoc CLI sys.path handling (GH-6419)
https://github.com/python/cpython/commit/d7ffa5820733a528d9ab87ee567738e2d3fd7126


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[miss-islington]

Change by miss-islington :


--
pull_requests: +6176
stage: commit review -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Nick Coghlan]

Change by Nick Coghlan :


--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Nick Coghlan]

Change by Nick Coghlan :


--
keywords: +patch
pull_requests: +6122
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Ned Deily]

Change by Ned Deily :


--
priority: critical -> release blocker
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Nick Coghlan]

Nick Coghlan  added the comment:

It turns out pydoc's CLI is looking specifically for '' in sys.path, but *not* 
looking for "os.getcwd()".

The resulting sys.path manipulation then goes completely wrong for "python -m 
pydoc", since it ends up removing "scriptdir" from sys.path, and that's the 
directory containing "pydoc" and the rest of the standard library.

It surprised me that you were getting the misbehaviour for the direct pydoc 
command though, as for me, that's a separate script living in `*/bin`, so the 
misbehaviour doesn't happen (only the bin directory gets removed from sys.path, 
which is the desired behaviour).

It turns out that behaviour is due to a difference between venv and virtualenv, 
where the latter defines a shell alias to hide the filesystem level pydoc 
scripts installed system wide:

```
$ which pydoc   

pydoc ()


{   


python -m pydoc "$@"


}
```

and hence will encounter the problem.

So I think there are two changes needed here:

1. Update pydoc.cli() to search sys.path for "os.getcwd()" in addition to 
searching for the empty string.
2. Rewording the note in the porting guide to explicitly call out checks for 
"'' in sys.path" in start-up code as potentially requiring updates

--
assignee:  -> ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Nick Coghlan]

Nick Coghlan  added the comment:

I guess this confirms our suspicion from issue 33053 that making "-m" no longer 
track the current working directory had the potential to pose some non-trivial 
compatibility risks :(

I can reproduce the issue in a 3.7 checkout without a virtual environment:

```
$ ./python -m pydoc itertools
No module named 'ast'
```

There's an odd discrepancy between the behaviour when running pydoc and the 
behaviour when running runpy, though:

```
$ ./python -im runpy
No module specified for execution
>>> import sys; print(len(sys.path)); print(sys.path)
5
['/home/ncoghlan/devel/py37', '/usr/local/lib/python37.zip', 
'/home/ncoghlan/devel/py37/Lib', 
'/home/ncoghlan/devel/py37/build/lib.linux-x86_64-3.7', 
'/home/ncoghlan/.local/lib/python3.7/site-packages']
>>>

$ ./python -im pydoc itertools
No module named 'ast'

>>> import sys; print(len(sys.path)); print(sys.path)
5
['.', '/home/ncoghlan/devel/py37', '/usr/local/lib/python37.zip', 
'/home/ncoghlan/devel/py37/build/lib.linux-x86_64-3.7', 
'/home/ncoghlan/.local/lib/python3.7/site-packages']
>>>
```

The path in the `runpy` case is correct, while the path in the `pydoc` case is 
missing the expected `Lib` directory (and hence can't import standard library 
modules), and has had a `.` inserted at the start of `sys.path`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Ned Deily]

Change by Ned Deily :


--
nosy: +ned.deily
priority: normal -> critical

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Geoffrey Spear]

Geoffrey Spear  added the comment:

This behavior seems to have been introduced in commit 
ee3784594b33c72c3fdca6a71892d22f14045ab6

--
nosy: +geoffreyspear, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Ned Batchelder]

Ned Batchelder  added the comment:

Oh, sorry, I forgot that import attempts aren't shown until -vv

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Ned Batchelder]

Ned Batchelder  added the comment:

Turns out it's even simpler:

$ pydoc itertools
No module named 'ast'

# !!!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33185] Python 3.7.0b3 fails in pydoc where b2 did not.

[Ned Batchelder]

New submission from Ned Batchelder :

"pydoc coverage" worked with 3.7b2, but fails with a surprising 
ModuleNotFoundError for configparser with b3.  The configparser is importable 
in the Python interpreter.  I tried with -v to what imports were attempted, and 
configparser isn't even mentioned until the failure message.

Complete reproduction:

# Using 3.7.0b2
$ mktmpenv -p /usr/local/pythonz/pythons/CPython-3.7.0b2/bin/python3.7 -n -q
/Library/Python/2.7/site-packages/virtualenv.py:1098: DeprecationWarning: the 
imp module is deprecated in favour of importlib; see the module's documentation 
for alternative uses
  import imp
Collecting pip
  Using cached pip-9.0.3-py2.py3-none-any.whl
Collecting setuptools
  Using cached setuptools-39.0.1-py2.py3-none-any.whl
Installing collected packages: pip, setuptools
  Found existing installation: pip 7.1.2
Uninstalling pip-7.1.2:
  Successfully uninstalled pip-7.1.2
  Found existing installation: setuptools 18.2
Uninstalling setuptools-18.2:
  Successfully uninstalled setuptools-18.2
Successfully installed pip-9.0.3 setuptools-39.0.1
This is a temporary environment. It will be deleted when you run 'deactivate'.
$ python -V
Python 3.7.0b2
$ pip install -q coverage==4.5.1
$ pydoc coverage
 Help on package coverage:

 NAME
 coverage - Code coverage measurement for Python.

 DESCRIPTION
 Ned Batchelder
 https://nedbatchelder.com/code/coverage

 PACKAGE CONTENTS
 __main__
 annotate
 backunittest
 backward
 bytecode
 cmdline
 collector
 config
 control
 data
 debug
 env
 execfile
 files
 html
 misc
 multiproc
 parser
 phystokens
 pickle2json
 plugin
 plugin_support
 python
 pytracer
 report
 results
 summary
 templite
 tracer
 version
 xmlreport

 DATA
 __url__ = 'https://coverage.readthedocs.io'
 version_info = (4, 5, 1, 'final', 0)

 VERSION
 4.5.1

 FILE
 
/usr/local/virtualenvs/tmp-e3c595a6301312d/lib/python3.7/site-packages/coverage/__init__.py


$ deactivate
Removing temporary environment: tmp-e3c595a6301312d
Removing tmp-e3c595a6301312d...

# Using 3.7.0b3
$ mktmpenv -p /usr/local/pythonz/pythons/CPython-3.7.0b3/bin/python3.7 -n -q
/Library/Python/2.7/site-packages/virtualenv.py:1098: DeprecationWarning: the 
imp module is deprecated in favour of importlib; see the module's documentation 
for alternative uses
  import imp
Collecting pip
  Using cached pip-9.0.3-py2.py3-none-any.whl
Collecting setuptools
  Using cached setuptools-39.0.1-py2.py3-none-any.whl
Installing collected packages: pip, setuptools
  Found existing installation: pip 7.1.2
Uninstalling pip-7.1.2:
  Successfully uninstalled pip-7.1.2
  Found existing installation: setuptools 18.2
Uninstalling setuptools-18.2:
  Successfully uninstalled setuptools-18.2
Successfully installed pip-9.0.3 setuptools-39.0.1
This is a temporary environment. It will be deleted when you run 'deactivate'.
$ python -V
Python 3.7.0b3
$ pip install -q coverage==4.5.1
$ pydoc coverage
problem in coverage - ModuleNotFoundError: No module named 'configparser'

$ python -c 'import configparser; print(configparser)'


$ python -v -m pydoc coverage
import _frozen_importlib # frozen
import _imp # builtin
import '_thread' # 
import '_warnings' # 
import '_weakref' # 
import '_frozen_importlib_external' # 
import '_io' # 
import 'marshal' # 
import 'posix' # 
import _thread # previously loaded ('_thread')
import '_thread' # 
import _weakref # previously loaded ('_weakref')
import '_weakref' # 
# installing zipimport hook
import 'zipimport' # 
# installed zipimport hook
import _thread # previously loaded ('_thread')
import '_thread' # 
import _weakref # previously loaded ('_weakref')
import '_weakref' # 
# 
/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc
 matches 
/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/encodings/__init__.py
# code object from 
'/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc'
# 
/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/__pycache__/codecs.cpython-37.pyc
 matches /usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/codecs.py
# code object from 
'/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/__pycache__/codecs.cpython-37.pyc'
import '_codecs' # 
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 
0x1005da438>
# 
/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc
 matches 
/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/encodings/aliases.py
# code object from 
'/usr/local/virtualenvs/tmp-95cdcd5f6881b25d/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader 
object at 0x1005e7f60>
import