[issue44170] ShareableList cannot safely handle multibyte utf-8 characters

2021-05-18 Thread Huw Jones
Huw Jones added the comment: The workaround I am using is to manually encode/decode. For the MWE, this means encoding on the creation side ``` shared_list = smm.ShareableList([s.encode() for s in strings]) ``` and decoding before using the string ``` for enc_str in shared_list: string

[issue44170] ShareableList cannot safely handle multibyte utf-8 characters

2021-05-18 Thread Huw Jones
New submission from Huw Jones : I've experienced a UnicodeDecodeError when adding unicode strings that contain multibye utf-8 characters into a shareable list. My observation is that ShareableList chunks the list of strings before sending it over the process boundary, however this chunking