Joshua Landau added the comment:

This doesn't seem to be a complete fix; the regex used does not include 
Other_ID_Start or Other_ID_Continue from

https://docs.python.org/3.5/reference/lexical_analysis.html#identifiers

Hence tokenize does not accept '℘·'.

Credit to modchan from http://stackoverflow.com/a/29586366/1763356.

----------
nosy: +Joshua.Landau
versions: +Python 3.4

_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue9712>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to