Inada Naoki <songofaca...@gmail.com> added the comment:
I succeeded to reproduce it on Ubuntu 20.04. $ sudo vi /var/lib/locales/supported.d/ja # add "ja_JP.EUC-JP EUC-JP" $ sudo locale-gen ja_JP.EUC-JP Generating locales (this might take a while)... ja_JP.EUC-JP... done Generation complete. $ chmod -r./build/lib.linux-x86_64-3.10/_sha3.cpython-310-x86_64-linux-gnu.so $ LC_ALL=ja_JP.eucjp ./python Python 3.10.0a0 (heads/master:fbf43f051e, Aug 17 2020, 15:13:52) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.setlocale(locale.LC_ALL, "") 'ja_JP.eucjp' >>> import _sha3 Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 101: invalid start byte Error message contains file path (byte string, probably encoded with fs encoding) and translated error message (encoded with locale encoding). I want to use "backslashescape" error handler, but both of PyUnicode_DecodeLocale() and PyUnicode_DecodeFSDefault() don't support it. After thinking about this several minutes, now I prefer PyUnicode_DecodeUTF8(msg, strlen(msg), "backslashreplace"). It fixes the issue with minimum behavior change, although error message is still backslashescaped. It might be the best practice for creating Unicode object from C error message like strerror(3). ---------- nosy: +inada.naoki _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue41894> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com