[
https://issues.apache.org/jira/browse/HCATALOG-630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Vasilenko updated HCATALOG-630:
--------------------------------------
Description:
Steps to replicate:
1) Create a Hive table
CREATE TABLE test_map (test map<STRING,INT> )
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;
2) Load the data:
LOAD DATA LOCAL INPATH '/home/dvasilen/Downloads/QA_TEST/test_map.json'
OVERWRITE INTO TABLE test_map;
3) The test_map.json contains the single line:
{"test":{"key":1}}
4) Run "select * from test_map;" from Hive client
5) Note the following exception being thrown:
13/03/13 13:34:20 ERROR CliDriver: Failed with exception
java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
java.io.IOException: org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:466)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1387)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:270)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:600)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:171)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:506)
... 13 more
Caused by: org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME)
not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
at
org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
at
org.codehaus.jackson.impl.JsonNumericParserBase._parseNumericValue(JsonNumericParserBase.java:399)
at
org.codehaus.jackson.impl.JsonNumericParserBase.getIntValue(JsonNumericParserBase.java:254)
at
org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:218)
at
org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:279)
at org.apache.hcatalog.data.JsonSerDe.populateRecord(JsonSerDe.java:188)
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:167)
... 14 more
If you change the value type to string
CREATE TABLE test_map (test map<STRING,STRING> )
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;
and
{"test":{"key":"string"}}
then it works correctly.
I see the same issue in 0.4.0 as well as 0.5.0.
was:
Steps to replicate:
1) Create a Hive table
CREATE TABLE test_map (test map<STRING,INT>)
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;
2) Load the data:
LOAD DATA LOCAL INPATH '/home/dvasilen/Downloads/QA_TEST/test_map.json'
OVERWRITE INTO TABLE test_map;
3) The test_map.json contains the single line:
{"test":{"key":1}}
4) Run "select * from test_map;" from Hive client
5) Note the following exception being thrown:
13/03/13 13:34:20 ERROR CliDriver: Failed with exception
java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
java.io.IOException: org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:466)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1387)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:270)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:600)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.hadoop.hive.serde2.SerDeException:
org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:171)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:506)
... 13 more
Caused by: org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME)
not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
at
org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
at
org.codehaus.jackson.impl.JsonNumericParserBase._parseNumericValue(JsonNumericParserBase.java:399)
at
org.codehaus.jackson.impl.JsonNumericParserBase.getIntValue(JsonNumericParserBase.java:254)
at
org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:218)
at
org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:279)
at org.apache.hcatalog.data.JsonSerDe.populateRecord(JsonSerDe.java:188)
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:167)
... 14 more
If you change the value type to string
CREATE TABLE test_map (test map<STRING,STRING>)
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;
and
{"test":{"key":"string"}}
then it works correctly.
I see the same issue in 0.4.0 as well as 0.5.0.
> org.apache.hcatalog.data.JsonSerDe cannot handle maps with numeric values
> --------------------------------------------------------------------------
>
> Key: HCATALOG-630
> URL: https://issues.apache.org/jira/browse/HCATALOG-630
> Project: HCatalog
> Issue Type: Bug
> Components: storage handlers
> Affects Versions: 0.4, 0.5
> Reporter: Dmitry Vasilenko
> Labels: JsonSerDe
>
> Steps to replicate:
> 1) Create a Hive table
> CREATE TABLE test_map (test map<STRING,INT> )
> ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
> STORED AS TEXTFILE;
> 2) Load the data:
> LOAD DATA LOCAL INPATH '/home/dvasilen/Downloads/QA_TEST/test_map.json'
> OVERWRITE INTO TABLE test_map;
> 3) The test_map.json contains the single line:
> {"test":{"key":1}}
> 4) Run "select * from test_map;" from Hive client
> 5) Note the following exception being thrown:
> 13/03/13 13:34:20 ERROR CliDriver: Failed with exception
> java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException:
> org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
> numeric, can not use numeric value accessors
> at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
> java.io.IOException: org.apache.hadoop.hive.serde2.SerDeException:
> org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
> numeric, can not use numeric value accessors
> at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:466)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1387)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:270)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:600)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.apache.hadoop.hive.serde2.SerDeException:
> org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not
> numeric, can not use numeric value accessors
> at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
> at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:171)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:506)
> ... 13 more
> Caused by: org.codehaus.jackson.JsonParseException: Current token
> (FIELD_NAME) not numeric, can not use numeric value accessors
> at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
> at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
> at
> org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
> at
> org.codehaus.jackson.impl.JsonNumericParserBase._parseNumericValue(JsonNumericParserBase.java:399)
> at
> org.codehaus.jackson.impl.JsonNumericParserBase.getIntValue(JsonNumericParserBase.java:254)
> at
> org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:218)
> at
> org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:279)
> at org.apache.hcatalog.data.JsonSerDe.populateRecord(JsonSerDe.java:188)
> at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:167)
> ... 14 more
>
> If you change the value type to string
> CREATE TABLE test_map (test map<STRING,STRING> )
> ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
> STORED AS TEXTFILE;
> and
> {"test":{"key":"string"}}
> then it works correctly.
> I see the same issue in 0.4.0 as well as 0.5.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
