[issue25758] ensurepip/venv broken on Windows if path includes unicode

[Steve Dower]

Steve Dower added the comment:

Applied Eryk's patch and updated the test to repro it (though it wouldn't repro 
on 3.6 with PEP 529 applied, but it definitely did on 3.5).

--
assignee:  -> steve.dower
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[Roundup Robot]

Roundup Robot added the comment:

New changeset 663a62bcf9c9 by Steve Dower in branch '3.5':
Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch 
by Eryk Sun)
https://hg.python.org/cpython/rev/663a62bcf9c9

New changeset ead30e7262d5 by Steve Dower in branch 'default':
Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch 
by Eryk Sun)
https://hg.python.org/cpython/rev/ead30e7262d5

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[Steve Dower]

Steve Dower added the comment:

Given a quick read, it looks like issue27781 (PEP 529) will resolve this?

Not encoding the path at all is obviously better, but maybe I'll add this as 
supporting evidence to the PEP...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[Nick Coghlan]

Nick Coghlan added the comment:

As a test case for handling non-ASCII characters in the name of the zipfile 
itself, I believe it should be sufficient to add 'Ł' to TEMP_DIR and TEMP_ZIP 
in https://hg.python.org/cpython/file/default/Lib/test/test_zipimport.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[John Mark Vandenberg]

Changes by John Mark Vandenberg :


--
nosy: +jayvdb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[Laura Creighton]

Changes by Laura Creighton :


--
nosy: +Marcus.Smith, dstufft, ncoghlan, paul.moore

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[Dima Tisnek]

New submission from Dima Tisnek:

One of my students installed Python 3.5 on Windows 10 to default location where 
user name "Łukasz" contains unicode.

Now "-m venv" and "-m ensurepip" do not work:

C:\Users\Łukasz>C:\Users\Łukasz\AppData\Local\Programs\Python\Python35-32\python.exe
 -m venv workshops
Error: Command '['C:\\Users\\\u0141ukasz\\workshops\\Scripts\\python.exe', 
'-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit 
status 1

C:\Users\Łukasz>C:\Users\Łukasz\AppData\Local\Programs\Python\Python35-32\python.exe
 -m ensurepip
Traceback (most recent call last):
  File 
"C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", 
line 170, in _run_module_as_main
"__main__", mod_spec)
  File 
"C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", 
line 85, in _run_code
exec(code, run_globals)
  File 
"C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__main__.py",
 line 4, in 
ensurepip._main()
  File 
"C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py",
 line 209, in _main
default_pip=args.default_pip,
  File 
"C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py",
 line 116, in bootstrap
_run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File 
"C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py",
 line 40, in _run_pip
import pip
  File "", line 969, in _find_and_load
  File "", line 954, in _find_and_load_unlocked
  File "", line 896, in _find_spec
  File "", line 1136, in find_spec
  File "", line 1112, in _get_spec
  File "", line 1093, in _legacy_get_spec
  File "", line 444, in spec_from_loader
  File "", line 530, in 
spec_from_file_location
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character

--
components: Unicode
messages: 255534
nosy: Dima.Tisnek, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: ensurepip/venv broken on Windows if path includes unicode
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[SilentGhost]

Changes by SilentGhost :


--
components: +Windows
nosy: +brett.cannon, eric.snow, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25758] ensurepip/venv broken on Windows if path includes unicode

[Eryk Sun]

Eryk Sun added the comment:

The problem is that the compile_source function in Modules/zipimport.c calls 
PyUnicode_EncodeFSDefault to get an encoded string to pass as the filename 
argument of Py_CompileString. On Windows this uses the ANSI codepage (i.e. 
'mbcs'). Apparently your system's ANSI codepage doesn't map the "Ł" character. 

I reproduced the problem more simply by copying pip-7.1.2-py2.py3-none-any.whl 
to a subdirectory named "Łukasz"; adding the wheel path to sys.path; and 
attempting to execute "import pip". 

One solution is to replace Py_CompileString with Py_CompileStringObject. This 
way compile_source doesn't have to worry about encoding its pathname argument. 
A minimal patch is attached, but it needs a test.

--
keywords: +patch
nosy: +eryksun
versions: +Python 3.6
Added file: http://bugs.python.org/file41185/issue25758_1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com