New submission from Huw Jones <h...@huwcbjones.co.uk>: I've experienced a UnicodeDecodeError when adding unicode strings that contain multibye utf-8 characters into a shareable list. My observation is that ShareableList chunks the list of strings before sending it over the process boundary, however this chunking process is not multibyte aware and will chunk in the middle of multibyte characters. On the other end, this results in the ShareableList throwing a UnicodeDecodeError when it fails to decode not-a-full multibyte utf-8 character.
>From running the attached MWE, I see that the string is sent in two chunks, >the first being b'Boom \xf0\x9f\x92\xa5 \xf0\x9f\x92\xa5 \xf0' which clearly >splits the 4 bytes of the 💥 character into the first byte and remaining 3 >bytes. ---------- components: Library (Lib) files: shareablelist_mwe.py messages: 393868 nosy: huwcbjones priority: normal severity: normal status: open title: ShareableList cannot safely handle multibyte utf-8 characters type: crash versions: Python 3.8 Added file: https://bugs.python.org/file50049/shareablelist_mwe.py _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue44170> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
