[issue31672] string.Template should use re.ASCII flag

Serhiy Storchaka added the comment: Results. There are 4 cases: 1) default idpattern and flags 2) overridden only idpattern 3) overridden only flags 4) overridden both idpattern and flags The case 1 was the one that was broken when this issue was opened. The initial Inada's version fixed the

[issue31672] string.Template should use re.ASCII flag

Serhiy Storchaka added the comment: New changeset 87be28f4a1c5b76926c71a3d9f92503f9eb82d51 by Serhiy Storchaka in branch 'master': bpo-31672: Restore the former behavior when override flags in Template. (#5099) https://github.com/python/cpython/commit/87be28f4a1c5b76926c71a3d9f92503f9eb82d51

[issue31672] string.Template should use re.ASCII flag

Serhiy Storchaka added the comment: PR 5099 reverts in 3.7 a subtle behavior change made by the fix for this issue. Before merging the fix setting flags to 0 in a Template subclass made the default pattern matching only lower case letters. Currently setting flags to 0 doesn't have any effect.

[issue31672] string.Template should use re.ASCII flag

Change by Serhiy Storchaka : -- pull_requests: +4967 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://m

[issue31672] string.Template should use re.ASCII flag

Change by Barry A. Warsaw : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ __

[issue31672] string.Template should use re.ASCII flag

Barry A. Warsaw added the comment: New changeset e256b408889eba867e1d90e5e1a0904843256255 by Barry Warsaw in branch 'master': bpo-31672 - Add one last minor clarification for idpattern (#4483) https://github.com/python/cpython/commit/e256b408889eba867e1d90e5e1a0904843256255 -- _

[issue31672] string.Template should use re.ASCII flag

Barry A. Warsaw added the comment: @inada.naoki - I think changing this to ?a will make things more clear. I submitted a PR for that, and then once it lands, we can close this issue. -- ___ Python tracker _

[issue31672] string.Template should use re.ASCII flag

Change by Barry A. Warsaw : -- pull_requests: +4420 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://ma

[issue31672] string.Template should use re.ASCII flag

Serhiy Storchaka added the comment: Is something left to do here? In 3.7 r'(?-i:[_a-zA-Z][_a-zA-Z0-9]*)' can be replaced with r'(?a:[_a-z][_a-z0-9]*)' if it will add clarity. -- ___ Python tracker _

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: New changeset 073150db39408c1800e4b9e895ad0b0e195f1056 by INADA Naoki in branch 'master': bpo-31672: doc: Remove one sentence from library/string.rst (GH-3990) https://github.com/python/cpython/commit/073150db39408c1800e4b9e895ad0b0e195f1056 -- ___

[issue31672] string.Template should use re.ASCII flag

Change by INADA Naoki : -- pull_requests: +3966 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: New changeset 7060380d577690a40ebc201c0725076349e977cd by INADA Naoki in branch '3.6': bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers (GH-3872) https://github.com/python/cpython/commit/7060380d577690a40ebc201c0725076349e977cd ---

[issue31672] string.Template should use re.ASCII flag

Change by INADA Naoki : -- pull_requests: +3958 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: New changeset b22273ec5d1992b0cbe078b887427ae9977dfb78 by INADA Naoki in branch 'master': bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers (GH-3872) https://github.com/python/cpython/commit/b22273ec5d1992b0cbe078b887427ae9977dfb78

[issue31672] string.Template should use re.ASCII flag

Barry A. Warsaw added the comment: With some comments to clarify of course. -- ___ Python tracker ___ ___ Python-bugs-list mailing l

[issue31672] string.Template should use re.ASCII flag

Barry A. Warsaw added the comment: On Oct 6, 2017, at 15:04, Serhiy Storchaka wrote: > Serhiy Storchaka added the comment: > > Another solution (works in 3.6 too): set idpattern to > r'(?-i:[_a-zA-Z][_a-zA-Z0-9]*)'. > > This looks pretty weird, setting the re.IGNORECASE flag and then unsett

[issue31672] string.Template should use re.ASCII flag

Serhiy Storchaka added the comment: Another solution (works in 3.6 too): set idpattern to r'(?-i:[_a-zA-Z][_a-zA-Z0-9]*)'. This looks pretty weird, setting the re.IGNORECASE flag and then unsetting it. But it works, and don't break the user code that changes idpattern without changing flags.

[issue31672] string.Template should use re.ASCII flag

Barry A. Warsaw added the comment: On Oct 4, 2017, at 10:05, Serhiy Storchaka wrote: > > See issue31690. But this solution can be used only in 3.7. That’s fine. I don’t think this is important enough to backport. -- ___ Python tracker

[issue31672] string.Template should use re.ASCII flag

Serhiy Storchaka added the comment: See issue31690. But this solution can be used only in 3.7. -- ___ Python tracker ___ ___ Python-

[issue31672] string.Template should use re.ASCII flag

Barry A. Warsaw added the comment: On Oct 4, 2017, at 02:29, INADA Naoki wrote: > > INADA Naoki added the comment: > >> Yet one way -- make re.ASCII a local flag. Than we could just change the >> idpattern attribute to r'(?a:[_a-z][_a-z0-9]*)', without touching the flags >> attribute. > >

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: Current pull request override `Template.flags = re.I` after class creation for backward compatibility, without any API change. But I'm not sure it's right approach. How many people who subclass string.Template expect non-ASCII match? If this change is bugfix for

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: > Yet one way -- make re.ASCII a local flag. Than we could just change the > idpattern attribute to r'(?a:[_a-z][_a-z0-9]*)', without touching the flags > attribute. https://docs.python.org/3.6/library/re.html#regular-expression-syntax says `(?imsx-imsx:...)` A

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: > When optimizing, please don't make API changes. This is not only optimization, but bugfix. Document of string.Template says: > By default, "identifier" is restricted to any case-insensitive ASCII > alphanumeric string (including underscores) that starts with

[issue31672] string.Template should use re.ASCII flag

Serhiy Storchaka added the comment: Yet one way -- make re.ASCII a local flag. Than we could just change the idpattern attribute to r'(?a:[_a-z][_a-z0-9]*)', without touching the flags attribute. -- ___ Python tracker

[issue31672] string.Template should use re.ASCII flag

Raymond Hettinger added the comment: When optimizing, please don't make API changes. -- nosy: +rhettinger ___ Python tracker ___ ___

[issue31672] string.Template should use re.ASCII flag

Change by INADA Naoki : -- stage: -> patch review type: -> behavior ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubs

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: > This means if someone does subclass string.Template and changes the pattern > to accept Unicode identifiers, then with this change they will also have to > modify flags, whereas before they didn't. Thank you for pointing it out. I removed re.A flag after compi

[issue31672] string.Template should use re.ASCII flag

Change by INADA Naoki : -- keywords: +patch pull_requests: +3851 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list

[issue31672] string.Template should use re.ASCII flag

Barry A. Warsaw added the comment: Technically it *is* an API change since `flags` is a part of the public API. The documentation says: $identifier names a substitution placeholder matching a mapping key of "identifier". By default, "identifier" is restricted to any case- insensi

[issue31672] string.Template should use re.ASCII flag

Change by Serhiy Storchaka : -- nosy: +barry, serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue31672] string.Template should use re.ASCII flag

New submission from INADA Naoki : Currently, strings.Template uses re.IGNORECASE without re.ASCII: idpattern = r'[_a-z][_a-z0-9]*' flags = _re.IGNORECASE [a-z] matches against 'ı' (0x131, DOTLESS I) and 'ſ' (0x17f, LONG S). It is not intentional, and it makes re.compile slower. ---