[
https://issues.apache.org/jira/browse/PIG-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Svinarchuk updated PIG-3627:
-----------------------------------
Attachment: PIG-3627.patch
> Json storage : Doesn't work in cases , where other Store Functions (like
> PigStorage / AvroStorage) do work.
> ------------------------------------------------------------------------------------------------------------
>
> Key: PIG-3627
> URL: https://issues.apache.org/jira/browse/PIG-3627
> Project: Pig
> Issue Type: Bug
> Reporter: jay vyas
> Attachments: PIG-3627.patch
>
>
> The following query
> {code:title=Bar.java|borderStyle=solid}
> pigServer.registerQuery(
> "uniqcnt = foreach transactionsG {"+
> "sym = transactions.product ;"+
> "dsym = distinct sym ;"+
> "generate flatten(dsym.product) as product,
> COUNT(dsym) as count ;" +
> "};");
> {code}
> Results in the schema:
> {code}
> Schema : {product: NULL,count: long}
> {code}
> This schema, is storable using AvroStorage or PigStorage, but it fails if
> stored using JsonStorage:
> {code}
> Failed to parse: <line 1, column 8> Syntax error, unexpected symbol at or
> near ','
> at
> org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:94)
> at
> org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:108)
> at org.apache.pig.impl.util.Utils.parseSchema(Utils.java:208)
> at org.apache.pig.impl.util.Utils.getSchemaFromString(Utils.java:182)
> at
> org.apache.pig.builtin.JsonStorage.prepareToWrite(JsonStorage.java:140)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
> {code}
> It appears that JsonStorage is thus less robust than the other storage
> formats. Can we confirm or deny if some types of data structures do/ do not
> work with JsonStorage?
> So,I suggest:
> 1) Ideally, I would think JsonStorage should support the same data that other
> Storage functions support.
> the next best thing:
> 2) Maybe a wiki page of examples that can / cannot work with JsonStorage
> and/or a better error message would be sufficient to solve this "bug".
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
