Dmitry Vasilenko created HCATALOG-630:
-----------------------------------------
Summary: org.apache.hcatalog.data.JsonSerDe cannot handle maps
with numeric values
Key: HCATALOG-630
URL: https://issues.apache.org/jira/browse/HCATALOG-630
Project: HCatalog
Issue Type: Bug
Components: storage handlers
Affects Versions: 0.5, 0.4
Reporter: Dmitry Vasilenko
Steps to replicate:
1) Create a Hive table
CREATE TABLE test_map (test map<STRING,INT>)
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;
2) Load the data:
LOAD DATA LOCAL INPATH '/home/dvasilen/Downloads/QA_TEST/test_map.json'
OVERWRITE INTO TABLE test_map;
3) The test_map.json contains the single line:
{"test":{"key":1}}
4) Run "select * from test_map;" from Hive client
5) Note the following exception being thrown:
13/03/13 13:34:20 ERROR CliDriver: Failed with exception
java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
java.io.IOException: org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:466)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1387)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:270)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:600)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:171)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:506)
... 13 more
Caused by: org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME)
not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
at
org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
at
org.codehaus.jackson.impl.JsonNumericParserBase._parseNumericValue(JsonNumericParserBase.java:399)
at
org.codehaus.jackson.impl.JsonNumericParserBase.getIntValue(JsonNumericParserBase.java:254)
at
org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:218)
at
org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:279)
at org.apache.hcatalog.data.JsonSerDe.populateRecord(JsonSerDe.java:188)
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:167)
... 14 more
If you change the value type to string
CREATE TABLE test_map (test map<STRING,STRING>)
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;
and
{"test":{"key":"string"}}
then it works correctly.
I see the same issue in 0.4.0 as well as 0.5.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
