[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: I don't understand the relationship with bpo-20087, so I removed the dependency. I fixed test_re in 3.7 and master branches. I close the issue. -- dependencies: -Mismatch between glibc and X11 locale.alias resolution: -> fixed stage: patch review

[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: New changeset 279657bac2856039ba422c18a3d7f227b455e9d6 by Victor Stinner in branch '3.7': [3.7] bpo-29571: Fix test_re.test_locale_flag() (GH-12178) https://github.com/python/cpython/commit/279657bac2856039ba422c18a3d7f227b455e9d6 --

[issue29571] test_re is failing when local is set for `en_IN`

Change by STINNER Victor : -- pull_requests: +12173 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue29571] test_re is failing when local is set for `en_IN`

Change by STINNER Victor : Added file: https://bugs.python.org/file48189/_testcapi.patch ___ Python tracker ___ ___ Python-bugs-list

[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: I wrote C and Python code to check what is the effective encoding used by the LC_CTYPE locale before setlocale(LC_CTYPE, "") is called on Python 3.7. Result: Windows uses the Latin1 encoding. See attached files: _testcapi.patch + loc.py produced loc.log

[issue29571] test_re is failing when local is set for `en_IN`

Change by STINNER Victor : Added file: https://bugs.python.org/file48188/loc.log ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: AppVeyor failed on the backport to Python 3.7 of my fix: PR 12108. Ok, now I understand the bug in Python 3.7. locale.getlocale(locale.LC_CTYPE)[1] returns None because Python doesn't set LC_CTYPE to the user preferred locale. I'm not sure of which locale

[issue29571] test_re is failing when local is set for `en_IN`

Change by miss-islington : -- pull_requests: +12115 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: New changeset ab71f8b793f7b42853ccd2a127ae7720adc5bcb4 by Victor Stinner in branch 'master': bpo-29571: Fix test_re.test_locale_flag() (GH-12099) https://github.com/python/cpython/commit/ab71f8b793f7b42853ccd2a127ae7720adc5bcb4 --

[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: > This seems to have broken test_re on Windows, see > https://ci.appveyor.com/project/python/cpython/build/3.7.0a0.1 It seems like the ANSI code page is 1252 ("cp1252"). == CPython 3.7.0a0 (master:d31b28e16a2387d0251df948ef5d1b33d4357652, Mar 5 2017,

[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: Ah, I can reproduce the bug on Fedora 29 using "LANG=en_IN ./python -m test -v test_re". The problem is that locale.getlocale() is not reliable: it pretends that the locale encoding is ISO8859-1, whereas the real encoding is UTF-8: $ LANG=en_IN ./python

[issue29571] test_re is failing when local is set for `en_IN`

Change by STINNER Victor : -- pull_requests: +12106 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing

[issue29571] test_re is failing when local is set for `en_IN`

Karthikeyan Singaravelan added the comment: Similar issue reported on debian9.8 stretch with python 3.7.2 and en_IN : issue36134 -- nosy: +xtreak ___ Python tracker ___

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: I've also added Matthias and Barry to the cc list, in case this does turn out to be a Debian or Ubuntu specific quirk. Restating the problem, the issue is that test_locale_flag in test_re may fail for at least the en_IN locale, and we're

[issue29571] test_re is failing when local is set for `en_IN`

Jaysinh shukla added the comment: Hello Nick, At the devsprints of Pycon India 2017, a few participants were facing this bug. They all were from the Ubuntu land. I have switched to Gentoo distro. I am not facing this bug, but let me confirm from any Ubuntu user.

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: Hmm, this actually works for me on Fedora 27 even if I go back to 1b3d88eb33085e90af729c4c2f78b5ba1b942b1e, the commit just before the initially merged (and subsequently reverted) test change above. Unassigning, since I can't readily

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: Hmm, even though we reverted the original test_re based change, and the initial attempted fix for bpo-20087 was also reverted, I'm still not currently seeing the failure for: LANG=en_IN.utf8 ./python -m test -v test_re I do have the

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Naman Bhalla : -- nosy: +Naman-Bhalla ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Naman Bhalla : -- pull_requests: +2751 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Serhiy Storchaka : -- pull_requests: -1096 ___ Python tracker ___

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Donald Stufft : -- pull_requests: +1096 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Benjamin Peterson added the comment: New changeset 21a74312f2d1ddee71fade709af49d078085ec30 by Benjamin Peterson in branch 'master': Revert "bpo-29571: Use correct locale encoding in test_re (#149)" (#554) https://github.com/python/cpython/commit/21a74312f2d1ddee71fade709af49d078085ec30

[issue29571] test_re is failing when local is set for `en_IN`

Benjamin Peterson added the comment: New changeset 312f7dfb7c669fcfc43020951b7f8ff521200ad7 by Benjamin Peterson in branch '3.5': Revert "bpo-29571: Use correct locale encoding in test_re (#149)" (#554) (#556) https://github.com/python/cpython/commit/312f7dfb7c669fcfc43020951b7f8ff521200ad7

[issue29571] test_re is failing when local is set for `en_IN`

Benjamin Peterson added the comment: New changeset 6a4b04cd337347d074ae0140fb13dca5bd4b11ef by Benjamin Peterson in branch '3.6': Revert "bpo-29571: Use correct locale encoding in test_re (#149)" (#554) (#555) https://github.com/python/cpython/commit/6a4b04cd337347d074ae0140fb13dca5bd4b11ef

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Benjamin Peterson : -- pull_requests: +457 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Benjamin Peterson : -- pull_requests: +456 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Benjamin Peterson : -- pull_requests: +455 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: Thanks for the explanation - given that, I agree that simply reverting the attempted test-based fix and instead relying on the issue 20087 updates is the way to go. -- ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Serhiy Storchaka : -- dependencies: +Mismatch between glibc and X11 locale.alias ___ Python tracker ___

[issue29571] test_re is failing when local is set for `en_IN`

Benjamin Peterson added the comment: I don't see what's odd about it. re.LOCALE uses the C locale, which one obtains from locale.getlocale(). getpreferredencoding() is not documented to have anything to do with the C locale, and indeed on Windows it may be completely different. --

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: I'm with Serhiy on this one: if the "re" module isn't using locale.getpreferredencoding(), then there's something odd going on. It just sounds like the disconnect on Windows is the opposite of the one we hit on Linux without Benjamin's patch, perhaps due to the

[issue29571] test_re is failing when local is set for `en_IN`

Benjamin Peterson added the comment: getpreferredencoding() takes a completely different path on windows (returns a codepage) and isn't related to the C locale. -- ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Benjamin Peterson added the comment: But the test was never broken on windows. On Sun, Mar 5, 2017, at 23:54, Serhiy Storchaka wrote: > > Serhiy Storchaka added the comment: > > I'm not sure this will help on Windows. > > -- > > ___ > Python

[issue29571] test_re is failing when local is set for `en_IN`

Serhiy Storchaka added the comment: And I don't understand why my fix doesn't work on Windows. -- ___ Python tracker ___

[issue29571] test_re is failing when local is set for `en_IN`

Serhiy Storchaka added the comment: I'm not sure this will help on Windows. -- ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Benjamin Peterson added the comment: Yep, I think we should merge https://github.com/python/cpython/pull/422 and revert ncoghlan's change. -- nosy: +benjamin.peterson ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Serhiy Storchaka : -- components: +Windows nosy: +paul.moore, steve.dower, tim.golden resolution: fixed -> stage: resolved -> status: closed -> open ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Zachary Ware added the comment: This seems to have broken test_re on Windows, see https://ci.appveyor.com/project/python/cpython/build/3.7.0a0.1 I found this change to be the culprit via git bisect, unfortunately we didn't have any working CI on Windows (buildbots were otherwise broken) at

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Benjamin Peterson : -- pull_requests: +352 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Nick Coghlan : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: New changeset 760f596b6a4b5514afe35e521621f484aef35413 by GitHub in branch '3.5': [3.5] bpo-29571: Use correct locale encoding in test_re (#149) (#154) https://github.com/python/cpython/commit/760f596b6a4b5514afe35e521621f484aef35413 --

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: New changeset 0683d6889bd4430599d22e12e201b8e9c45be5a2 by GitHub in branch '3.6': [3.6] bpo-29571: Use correct locale encoding in test_re (#149) (#153) https://github.com/python/cpython/commit/0683d6889bd4430599d22e12e201b8e9c45be5a2 --

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Nick Coghlan : -- pull_requests: +118 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Nick Coghlan : -- pull_requests: +117 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: New changeset ace5c0fdd9b962e6e886c29dbcea72c53f051dc4 by GitHub in branch 'master': bpo-29571: Use correct locale encoding in test_re (#149) https://github.com/python/cpython/commit/ace5c0fdd9b962e6e886c29dbcea72c53f051dc4 --

[issue29571] test_re is failing when local is set for `en_IN`

Serhiy Storchaka added the comment: Yes, please push it Nick. -- ___ Python tracker ___ ___ Python-bugs-list

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Nick Coghlan : -- pull_requests: +112 ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: Looking into this at the PyCon Pune sprints, the problem appears to be arising due to the following difference in behaviour when the unqualifed `en_IN` locale is set: $ LANG=en_IN.UTF-8 python3 -c "import locale; print(locale.getlocale(locale.LC_CTYPE),

[issue29571] test_re is failing when local is set for `en_IN`

Nick Coghlan added the comment: I have a few folks hitting this at the PyCon Pune sprints, so I'm going to apply Serhiy's patch :) -- assignee: serhiy.storchaka -> ncoghlan nosy: +ncoghlan ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Serhiy Storchaka added the comment: > Seriously? Not a GitHub pull request? ;-) (old habit?) I'm not experienced with git, and devguide still looks not ready. -- ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Serhiy Storchaka added the comment: Good point. The test used locale.getlocale() and it returned returned ('en_IN', 'ISO8859-1'). Following patch makes the test using locale.getpreferredencoding(False), the same encoding as was reported at the header of test report. -- keywords:

[issue29571] test_re is failing when local is set for `en_IN`

STINNER Victor added the comment: > Following patch ... Seriously? Not a GitHub pull request? ;-) (old habit?) -- ___ Python tracker ___

[issue29571] test_re is failing when local is set for `en_IN`

Matthew Barnett added the comment: The report says "== encodings: locale=UTF-8, FS=utf-8". It says that "test_locale_caching" was skipped, but also that "test_locale_flag" failed. -- ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Serhiy Storchaka added the comment: Locale encoding is ISO8859-1. This test is skipped on non 8-bit locale. This is a problem with tests, not with the re module. I don't have a solution. -- components: +Tests versions: +Python 3.5, Python 3.6 ___

[issue29571] test_re is failing when local is set for `en_IN`

Matthew Barnett added the comment: I'm just wondering whether the problem is just due to the locale's encoding being UTF-8. The locale support in re really only works with encodings that use 1 byte/character. -- ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Changes by Serhiy Storchaka : -- assignee: -> serhiy.storchaka nosy: +serhiy.storchaka type: -> behavior ___ Python tracker

[issue29571] test_re is failing when local is set for `en_IN`

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___

[issue29571] test_re is failing when local is set for `en_IN`

New submission from Jaysinh shukla: Description: A test case is failing while running `./python -m test -v test_re`. Traceback: $>./python -m test -v test_re == CPython 3.7.0a0 (default, Feb 15 2017, 22:28:32) [GCC 5.4.0 20160609] == Linux-4.4.0-62-generic-x86_64-with-debian-stretch-sid