[ https://issues.apache.org/jira/browse/THRIFT-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904618#comment-16904618 ]
James E. King III commented on THRIFT-4677: ------------------------------------------- Thrift only handles strings as UTF8 internally. I think you will need to convert your string to UTF8 binary before you attempt to transmit it by Thrift. For example I used the online tool [https://mothereff.in/utf-8] and pasted in your string of '任务尚未运行。' and I got: {{\xE4\xBB\xBB\xE5\x8A\xA1\xE5\xB0\x9A\xE6\x9C\xAA\xE8\xBF\x90\xE8\xA1\x8C\xE3\x80\x82}} But it looks like the strings are not stored in UTF8 in your database, so between reading them out and putting them in to a thrift struct for transmission, convert them to UTF8. > UnicodeDecodeError in Python3 > ----------------------------- > > Key: THRIFT-4677 > URL: https://issues.apache.org/jira/browse/THRIFT-4677 > Project: Thrift > Issue Type: Bug > Components: Python - Library > Environment: Operating System: Windows 10 Pro (Simplified Chinese) > Python Interpreter: Python 3.6.6 > {{osquery}} Version: 3.3.0 > {{osquery-python}} Version: 3.0.5 > > Reporter: Jarry Shaw > Priority: Major > Attachments: compat.py, osquery_all_mp.py > > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > This is an issue occurred when using > [osquery-python|https://github.com/osquery/osquery-python] (Python binding of > [osquery|https://osquery.io/] by Facebook). > When querying, {{UnicodeDecodeError}} raised with error message: "{{'utf-8' > codec can't decode byte 0xc3 in position 0: invalid continuation byte}}" from > {{thrift.compat.binary_to_str}}, which is because the encoding of {{bin_val}} > parameter should be "{{gbk}}". > Possible approaches are: > * add a parameter for user to determine encodings > * get the system encoding through {{locale.getpreferredencoding()}} > * call {{bin_val.decode}} with {{errors='replace'}} or {{errors='ignore'}} > parameter > * introduce {{chardet}} to try and resolve encoding problems > The attachment is my hack solution to this issue (through not perfect). -- This message was sent by Atlassian JIRA (v7.6.14#76016)