[
https://issues.apache.org/jira/browse/IMPALA-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736503#comment-16736503
]
Paul Rogers edited comment on IMPALA-8051 at 1/8/19 12:09 AM:
--------------------------------------------------------------
Turns out that the "#" character has unique problems. It is a comment
character, which seems to fall through the cracks in the techniques we use to
detect that the identifier must be quoted. To check for quotes, we let Hive
parse the token. If the name is, say, "foo+", we get two tokens, and so quote
the name. But, for "foo#", we get just one token because the "#" starts a
comment.
Added this trivial fix to the patch for IMPALA-7905.
was (Author: paul.rogers):
Turns out that the "#" character has unique problems. It is a comment
character, which seems to fall through the cracks in the techniques we use to
detect that the identifier must be quoted. To check for quotes, we let Hive
parse the token. If the name is, say, "foo+", we get two tokens, and so quote
the name. But, for "foo#", we get just one token because the "#" starts a
comment.
> Compute stats fails on a column with comment character in name
> --------------------------------------------------------------
>
> Key: IMPALA-8051
> URL: https://issues.apache.org/jira/browse/IMPALA-8051
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.1.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Minor
>
> Problem - "compute stats" query executed on a table containing a special
> character "#" in one of its columns is failing with below error:
> WARNINGS: AnalysisException: Syntax error in line 1:
> ...length(cola)), NDV(colb#) AS colb#, CAST(-1 as BIG...
> ^
> Encountered: Unexpected character
> Expected: ADD, ALTER, AND, ARRAY, AS, ASC, BETWEEN, BIGINT, BINARY,
> BLOCK_SIZE, BOOLEAN, CACHED, CASCADE, CHANGE, CHAR, COMMENT, COMPRESSION,
> CROSS, DATE, DATETIME, DECIMAL, DEFAULT, DESC, DIV, REAL, DROP, ELSE,
> ENCODING, END, FLOAT, FOLLOWING, FROM, FULL, GROUP, IGNORE, HAVING, ILIKE,
> IN, INNER, INTEGER, IREGEXP, IS, JOIN, LEFT, LIKE, LIMIT, LOCATION, MAP, NOT,
> NULL, NULLS, OFFSET, ON, OR, ORDER, PARTITION, PARTITIONED, PRECEDING,
> PRIMARY, PURGE, RANGE, RECOVER, REGEXP, RENAME, REPLACE, RESTRICT, RIGHT,
> RLIKE, ROW, ROWS, SELECT, SET, SMALLINT, SORT, STORED, STRAIGHT_JOIN, STRING,
> STRUCT, TABLESAMPLE, TBLPROPERTIES, THEN, TIMESTAMP, TINYINT, TO, UNCACHED,
> UNION, USING, VALUES, VARCHAR, WHEN, WHERE, WITH, COMMA, IDENTIFIER
> Steps to reproduce the issue -
> # Create a table containing special character in one of it columns from Hive.
> For example:
> {code:sql}
> CREATE TABLE test_special_character (`id#` int);
> {code}
> # Execute "INVALIDATE METADATA test_special_character" from Impala.
> # Execute "COMPUTE STATS test_special_character" from Impala and it'll lead
> to above mentioned error.
> Impala does not allow to create tables with columns containing special
> characters but Hive allows it by using back ticks (``) to escape it. However,
> Impala still can load the metadata of table and can read from column
> containing special character as well by escaping the special character using
> back ticks (``). For example, below query can be executed from Impala -
> {code:sql}
> select `id#` from test_special_character;
> {code}
> However, when "compute stats" command is executed, the query triggered by
> this command to compute column-level stats does not use back ticks to escape
> the special character present in one of the columns as it does not know that
> column contains a special character and this is the cause of this issue.
> (Reported by a user.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
