Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16960
Change subject: IMPALA-10434: Fix impala-shell's unicode regressions on Python2 ...................................................................... IMPALA-10434: Fix impala-shell's unicode regressions on Python2 To make impala-shell compatible for Python3, we explicitly distinguish bytes and text in Python2 by decoding the bytes for all inputs. Regression 1: multiple queries in one line with unicode chars will break In precmd() of impala-shell, if there are multiple queries present in one input line, we split it into individual queries (by sqlparse.split()) and append them back to the 'cmdqueue'. They will be passed to precmd() again. In our Python2 implementation, precmd() expects them to be str type, and will decode them into unicode type. However, the output type of sqlparse.split() is unicode which doesn't have a decode() method. Calling decode() on a unicode var will let Python2 implicitly encode it to str. This may cause UnicodeEncodeError since implicitly encoding use 'ascii'. Regression 2: multi-line query with unicode chars will break when command history is enabled In _check_for_command_completion(), when calling readline.replace_history_item in Python2. We encode the completed_cmd into bytes. However, we shouldn't replace it since the return type is expected to be unicode. Tests: - Add tests for these two regressions in Python2. Change-Id: Icc4a8d31311a5c59e5fc0e65fe09f770df41bea4 --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 30 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16960/1 -- To view, visit http://gerrit.cloudera.org:8080/16960 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icc4a8d31311a5c59e5fc0e65fe09f770df41bea4 Gerrit-Change-Number: 16960 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
