[ https://issues.apache.org/jira/browse/IMPALA-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pooja Nilangekar resolved IMPALA-6223. -------------------------------------- Resolution: Fixed > Gracefully handle malformed 'with' queries in impala-shell > ---------------------------------------------------------- > > Key: IMPALA-6223 > URL: https://issues.apache.org/jira/browse/IMPALA-6223 > Project: IMPALA > Issue Type: Bug > Components: Clients > Affects Versions: Impala 2.10.0 > Reporter: bharath v > Assignee: Pooja Nilangekar > Priority: Minor > Labels: newbie > > Impala shell can throw a lexer error if it encounters a malformed "with" > query. > {noformat} > impala-shell.sh -q "with foo as (select bar from temp where temp.a='" > Starting Impala Shell without Kerberos authentication > Connected to localhost:21000 > Server version: impalad version 2.11.0-SNAPSHOT DEBUG (build > 0ee1765f38082bc5c10aa37b23cb8e57caa57d4e) > Traceback (most recent call last): > File "/home/bharath/Impala/shell/impala_shell.py", line 1463, in <module> > execute_queries_non_interactive_mode(options, query_options) > File "/home/bharath/Impala/shell/impala_shell.py", line 1338, in > execute_queries_non_interactive_mode > shell.execute_query_list(queries)): > File "/home/bharath/Impala/shell/impala_shell.py", line 1218, in > execute_query_list > if self.onecmd(q) is CmdStatus.ERROR: > File "/home/bharath/Impala/shell/impala_shell.py", line 505, in onecmd > return cmd.Cmd.onecmd(self, line) > File "/usr/lib/python2.7/cmd.py", line 221, in onecmd > return func(arg) > File "/home/bharath/Impala/shell/impala_shell.py", line 1024, in do_with > tokens = list(lexer) > File "/usr/lib/python2.7/shlex.py", line 269, in next > token = self.get_token() > File "/usr/lib/python2.7/shlex.py", line 96, in get_token > raw = self.read_token() > File "/usr/lib/python2.7/shlex.py", line 172, in read_token > raise ValueError, "No closing quotation" > ValueError: No closing quotation > {noformat} > This happens because we use shlex to parse the input query to determine if > its a DML and it can throw if the input doesn't have balanced quotes. > {noformat} > def do_with(self, args): > """Executes a query with a WITH clause, fetching all rows""" > query = self.imp_client.create_beeswax_query("with %s" % args, > self.set_query_options) > # Set posix=True and add "'" to escaped quotes > # to deal with escaped quotes in string literals > lexer = shlex.shlex(query.query.lstrip(), posix=True) > lexer.escapedquotes += "'" > # Because the WITH clause may precede DML or SELECT queries, > # just checking the first token is insufficient. > is_dml = False > tokens = list(lexer) <---- > {noformat} > A simple shlex repro of that is as follows, > {noformat} > >>> lexer = shlex.shlex("with foo as (select bar from temp where temp.a='", > >>> posix=True); > >>> list(lexer) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python2.7/shlex.py", line 269, in next > token = self.get_token() > File "/usr/lib/python2.7/shlex.py", line 96, in get_token > raw = self.read_token() > File "/usr/lib/python2.7/shlex.py", line 172, in read_token > raise ValueError, "No closing quotation" > ValueError: No closing quotation > {noformat} > Fix: Either catch the exception and handle it gracefully or have a better way > to figure out the query type, using a SQL parser (more involved). > This query also repros it: > {code} > with v as (select 1) > select foo('\\'), ('bar > ; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)