[
https://issues.apache.org/jira/browse/IMPALA-10051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172327#comment-17172327
]
Tamas Mate commented on IMPALA-10051:
-------------------------------------
In the ImpalaShell.do_with method shlex.split is used to tokenize the query so
that it can be decided whether the query is DML or not.
However, whitespace bound tokens in shlex by default. Therefore, whitespace in
a query string can cause the closure of a token prematurely before the final
quote could be evaluated. Which triggers the 'No closing quotation' exception.
In this example the space after the + sign causes the issue.
impala-shell.sh -q 'with select regexp_replace(column_name,"[a-zA-Z]","+ ");'
> impala-shell exits with ValueError with WITH clauses
> ----------------------------------------------------
>
> Key: IMPALA-10051
> URL: https://issues.apache.org/jira/browse/IMPALA-10051
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
> Affects Versions: Impala 4.0
> Reporter: Tamas Mate
> Assignee: Tamas Mate
> Priority: Major
>
> Some strings can cause shlex to throw an exception in WITH clauses, for
> example in a regexp_replace. This should be handled more gracefully and
> correctly.
> Working query (impala-shell forwards the query for analysis):
> {code:java}
> impala-shell.sh -q 'with select regexp_replace(column_name, "[a-zA-Z]", "+
> ");'
> {code}
> While same query fails with ValueError when empty spaces are removed from the
> arguments of the regexp_replace:
> {code:java}
> tmate@tmate-box:~/Projects/Impala$ impala-shell.sh -q 'with select
> regexp_replace(column_name,"[a-zA-Z]","+ ");'
> Starting Impala Shell with no authentication using Python 2.7.16
> Warning: live_progress only applies to interactive shell sessions, and is
> being skipped for now.
> Opened TCP connection to localhost:21000
> Connected to localhost:21000
> Server version: impalad version 4.0.0-SNAPSHOT DEBUG (build
> b29cb4ca82a4f05ea7dc0eadc330a64fbe685ef0)
> Traceback (most recent call last):
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1973, in
> <module>
> impala_shell_main()
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1927, in
> impala_shell_main
> if execute_queries_non_interactive_mode(options, query_options):
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1731, in
> execute_queries_non_interactive_mode
> shell.execute_query_list(queries))
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1564, in
> execute_query_list
> if self.onecmd(q) is CmdStatus.ERROR:
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 675, in
> onecmd
> return func(arg)
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1276, in
> do_with
> tokens = shlex.split(strip_comments(query.lstrip()), posix=False)
> File
> "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py",
> line 279, in split
> return list(lex)
> File
> "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py",
> line 269, in next
> token = self.get_token()
> File
> "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py",
> line 96, in get_token
> raw = self.read_token()
> File
> "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py",
> line 172, in read_token
> raise ValueError, "No closing quotation"
> ValueError: No closing quotation
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
