[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: > I thought you had a unit test, I don't see any in your commit "patch -p0 < ... && svn ci" doesn't include new files. I forgot it. r81361 includes the new file. -- ___ Python tracker

[issue8663] Failed compile in a non-ASCII path

Tarek Ziadé added the comment: I thought you had a unit test, I don't see any in your commit -- ___ Python tracker ___ ___ Python-bugs

[issue8663] Failed compile in a non-ASCII path

Changes by STINNER Victor : -- status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: > I commited to patch on distutils.log in r81359 (py3k) Backported to 3.1 as r81363. -- status: pending -> open ___ Python tracker ___

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: I commited to patch on distutils.log in r81359 (py3k). I'm waiting for the buildbot before backporting to 3.1. -- resolution: -> fixed status: open -> pending ___ Python tracker

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: > Although, all these patchs should have some tests demonstrating the bugs As discussed on IRC, fixing distutils.log package instead of setup.py is better. New patch includes unit tests. -- Added file: http://bugs.python.org/file17401/distutils_log_b

[issue8663] Failed compile in a non-ASCII path

Changes by STINNER Victor : Removed file: http://bugs.python.org/file17342/distutils_spawn_log.patch ___ Python tracker ___ ___ Python-bugs-lis

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: setup_stdout_backslashreplace.patch implements point (a). I don't know which solution is better. This issue is not specific to compiling CPython, other programs may fail in non-ASCII paths and so solution (b) is maybe better. -- Added file: http://bu

[issue8663] Failed compile in a non-ASCII path

Tarek Ziadé added the comment: I'll look at that asap. Although, all these patchs should have some tests demonstrating the bugs -- ___ Python tracker ___ ___

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: Conditions to reproduce the bug: - non ASCII directory - don't use make -j N (no MAKEFLAGS environment variable) - write make output into a pipe, eg. make 2>&1|cat My 2 last patches are not enough: there are other functions writing non-ASCII strings to log

[issue8663] Failed compile in a non-ASCII path

Changes by STINNER Victor : Removed file: http://bugs.python.org/file17268/distutils_spawn_toascii.patch ___ Python tracker ___ ___ Python-bugs

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: New patch: use sys.stdout.encoding instead of ASCII. (If stdout is not a TTY, sys.stdout.encoding is ASCII.) -- Added file: http://bugs.python.org/file17342/distutils_spawn_log.patch ___ Python tracker

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: I tried to recompile Python with "export PYTHONIOENCODING=ascii" but it doesn't fail. That's because the Makefile calls "./python -E ./setup.py -q build": -E ignores environment variables. -- ___ Python tracker

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: > > If the standard output is not a TTY, Python uses ASCII encoding > > for sys.stdout > We could perhaps fix this too, if python-dev agrees. Open a new issue please if you consider that as a bug. -- ___ Python tr

[issue8663] Failed compile in a non-ASCII path

Antoine Pitrou added the comment: > Wasn’t PYTHONIOENCODING added for such cases? Yes, it was, but it's a very bad workaround. In most if not all cases, people will set PYTHONIOENCODING to their system's default encoding. Therefore, they shouldn't have to set an environment variable at all. --

[issue8663] Failed compile in a non-ASCII path

Éric Araujo added the comment: Wasn’t PYTHONIOENCODING added for such cases? -- ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: Attached patch escapes non-ASCII characters of the log message using ASCII+backslashreplace (but keep unicode type). -- keywords: +patch Added file: http://bugs.python.org/file17268/distutils_spawn_toascii.patch ___

[issue8663] Failed compile in a non-ASCII path

Antoine Pitrou added the comment: > If the standard output is not a TTY, Python uses ASCII encoding for > sys.stdout We could perhaps fix this too, if python-dev agrees. -- ___ Python tracker

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: If the standard output is not a TTY, Python uses ASCII encoding for sys.stdout: ./python -c "import sys;print(sys.stdout.encoding)"|cat => ascii. This issue remembers me: #8533 (regrtest: use backslashreplace error handler for stdout). -- __

[issue8663] Failed compile in a non-ASCII path

Éric Araujo added the comment: (I wrote before I saw Victor’s reply) Does it work with PYTHONIOENDODING set to UTF-8? -- ___ Python tracker ___ _

[issue8663] Failed compile in a non-ASCII path

Éric Araujo added the comment: Another long-standing encoding bug was fixed: It was impossible to use e.g. an author name with non-ASCII characters. The fix makes distutils use UTF-8. Of course, it’s more complicated to choose an encoding for terminal I/O. -- nosy: +merwok __

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: Conditions to reproduce the bug: don't use make -j N (unset MAKEFLAGS), stdout and stderr should be be TTY => use make 2>&1|cat. -- ___ Python tracker

[issue8663] Failed compile in a non-ASCII path

Antoine Pitrou added the comment: Anyway, regardless of the actual stdout encoding, distutils should be able to log messages without crashing, IMO. -- ___ Python tracker ___ ___

[issue8663] Failed compile in a non-ASCII path

Antoine Pitrou added the comment: Ok, this might be more interesting: $ cat | ./python -i Python 3.2a0 (py3k:81010, May 8 2010, 23:25:47) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.getpreferredencoding() 'UTF-8' >>

[issue8663] Failed compile in a non-ASCII path

Antoine Pitrou added the comment: As I answered on IRC, this is on a buildbot environment. Compiling from the command line works fine, but not from the buildbot environment. >From the command line, the locale is as following: $ locale LANG=en_GB.UTF-8 LC_CTYPE="en_GB.UTF-8" LC_NUMERIC="en_GB.U

[issue8663] Failed compile in a non-ASCII path

STINNER Victor added the comment: What is your locale and your locale encoding? distutils use ASCII but I'm not sure that your locale encoding is ASCII, because Python fails to start if the locale is ASCII and the path contains a non ASCII character. See also issue #8611: "Python3 doesn't sup

[issue8663] Failed compile in a non-ASCII path

New submission from Antoine Pitrou : Distutils fails logging characters which are not part of the "default encoding": http://www.python.org/dev/buildbot/builders/AMD64%20Ubuntu%20wide%203.x/builds/1062/steps/compile/logs/stdio [...] File "/home/buildbot/cpython-ucs4-nonascii-\u20ac/3.x.pitro