[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Change by Serhiy Storchaka : -- pull_requests: -6086 ___ Python tracker ___

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Change by Ville Skyttä : -- pull_requests: +6086 ___ Python tracker ___ ___

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

STINNER Victor added the comment: New changeset 77bf6da7258b4a312e224860ea50ac010aa17c1e by Victor Stinner (Miss Islington (bot)) in branch '3.6': bpo-28393: Update encoding lookup docs wrt bpo-27938 (GH-4871) (#4881)

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Change by Roundup Robot : -- pull_requests: +4777 ___ Python tracker ___

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

STINNER Victor added the comment: New changeset 297fd876aad8ef443d8992618de22c46dbda258b by Victor Stinner (Ville Skyttä) in branch 'master': bpo-28393: Update encoding lookup docs wrt bpo-27938 (#4871)

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Change by Ville Skyttä : -- pull_requests: +4763 ___ Python tracker ___ ___

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Kubilay Kocak added the comment: @Victor I was just checking this issue to copy the test command, to provide results to you both when I saw the lovely surprise. Thank you :) -- ___ Python tracker

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

STINNER Victor added the comment: Sorry for the little breakage of FreeBSD buildbots, it seems to be ok now ;-) -- resolution: -> fixed status: open -> closed ___ Python tracker

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

STINNER Victor added the comment: > New changeset 3b185df3a3e2 by Victor Stinner in branch 'default': > Fix check_force_ascii() > https://hg.python.org/cpython/rev/3b185df3a3e2 @koobs: That's my tiny gift for your birthday. Happy Birthday! ;-) (It should fix FreeBSD buildbots.) --

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Roundup Robot added the comment: New changeset 3b185df3a3e2 by Victor Stinner in branch 'default': Fix check_force_ascii() https://hg.python.org/cpython/rev/3b185df3a3e2 -- ___ Python tracker

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

STINNER Victor added the comment: > FAIL: test_create_at_shutdown_without_encoding > (test.test_io.PyTextIOWrapperTest) Steve fixed it: --- changeset: 103229:47b4dbd451f5 tag: tip user:Steve Dower date:Wed Sep 07 09:31:52 2016 -0700 files:

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Steve Dower added the comment: The Windows buildbot failures are partly my fault and partly Ben's fault (I created a new error message - Ben added it to the wrong test), so I'll go and prevent the error message. No idea on the other issue. It doesn't repro for me, but since it seems to be

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Serhiy Storchaka added the comment: Maybe Windows buildbots failures are related: http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/8294/steps/test/logs/stdio == FAIL:

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

STINNER Victor added the comment: Sorry, but I don't have enough information to fix the issue. I don't see how my change can break the two failing tests. Could you please try to collect more information manually? -- status: closed -> open ___

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Martin Panter added the comment: Koobs if you can, it would be good to understand where the failure is. My guess is that Python doesn’t like running a non-ASCII filename. The following is hopefully a simplified version of the test_cmd_line_script test case: import os, subprocess, sys

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

koobs added the comment: Re-open and assign for regressions. Observed in all koobs-freebsd* buildbots (9/10/11) and build types. Issue is in default branch (add version 3.7) First failing test run: http://buildbot.python.org/all/builders/AMD64%20FreeBSD%20CURRENT%20Non-Debug%203.x/builds/110

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Martin Panter added the comment: It seems this change is the cause of the Free BSD buildbot failures. From memory, both failing cases involve sending or receiving non-ASCII bytes in child Python processes.

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

STINNER Victor added the comment: > See also get_standard_encoding() in Python/codecs.c. I suppose it is faster. I understand that PyCodec_SurrogatePassErrors() is already called with a normalized encoding name. With my enhanced _Py_normalize_encoding(), strange syntaxes like " utf 8 " also

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Roundup Robot added the comment: New changeset 99818330b4c0 by Victor Stinner in branch 'default': Issue #27938: Add a fast-path for us-ascii encoding https://hg.python.org/cpython/rev/99818330b4c0 -- nosy: +python-dev ___ Python tracker

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

Serhiy Storchaka added the comment: See also get_standard_encoding() in Python/codecs.c. I suppose it is faster. UTF-32 is rarely used as external encoding, but it is still used as internal encoding in some programming languages and libraries (e.g. wchar_t* in C and std::wstring in C++ on

[issue27938] PyUnicode_AsEncodedString, PyUnicode_Decode: add fast-path for "us-ascii" encoding

New submission from STINNER Victor: The "us-ascii" encoding is an alias to the Python ASCII encoding. PyUnicode_AsEncodedString() and PyUnicode_Decode() functions have a fast-path for the "ascii" string, but not for "us-ascii". Attached patch uses also the fast-path for "us-ascii". It's a