Darryl Dutton created HIVE-20628:
------------------------------------
Summary: Parsing error when using a complex map data type under
dynamic column masking
Key: HIVE-20628
URL: https://issues.apache.org/jira/browse/HIVE-20628
Project: Hive
Issue Type: Bug
Components: Hive, HiveServer2, Parser, Security
Affects Versions: 2.1.0
Environment: The error can be simulated using HDP 2.6.4 sandbox
Reporter: Darryl Dutton
When trying to use the map complex data type as part of dynamic column mask,
Hive throws a parsing error as it is expecting a primitive type (see trace
pasted below). The use case is trying to apply masking to elements within a map
type by applying a custom hive UDF (to apply the mask) using Ranger. Expect
Hive to support complex data types for masking in addition to the primitive
types. The expectation occurs when Hive need to evaluate the UDF or apply a
standard mask (pass-through works as expected). You can recreate the problem by
creating a simple table with a map data type column, then applying the masking
to that column through a Ranger resource based policy and a custom function
(you can use a standard Hive UDF str_to_map('F4','XXXX') to simulate returning
a map).
CREATE TABLE `mask_test`(
`key` string,
`value` map<string,string>)
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
INSERT INTO TABLE mask_test
SELECT 'AAA' as key,
map('F1','2022','F2','2222','F3','3333333') as value
FROM (select 1 ) as temp;
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.parse.SemanticException:org.apache.hadoop.hive.ql.parse.ParseException:
line 1:57 cannot recognize input near 'map' '<' 'string' in primitive type
specification
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(SemanticAnalyzer.java:10370)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10486)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:219)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:465)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:321)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1224)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1218)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:146)
... 15 more
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.parse.ParseException:line 1:57 cannot recognize input
near 'map' '<' 'string' in primitive type specification
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:214)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:171)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(SemanticAnalyzer.java:10368)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
