[
https://issues.apache.org/jira/browse/PIG-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Bhat updated PIG-3331:
----------------------------
Description:
Script which stores Avro using a predefined schema does not store the default
values in the file
{code}
a = LOAD 'numbers.txt' USING PigStorage (':') as (intnum1000: int,id:
int,intnum5: int,intnum100: int,intnum: int,longnum: long,floatnum:
float,doublenum: double);
b2 = foreach a generate id, intnum5, intnum100;
c2 = filter b2 by 110 <= id and id < 120;
STORE c2 INTO '/tmp/TestAvroStorage/testDefaultValueWrite' USING
org.apache.pig.piggybank.storage.avro.AvroStorage (' { "debug" : 5, "schema" :
{ "name" : "rmyrecord", "type" : "record", "fields" : [ { "name" : "id",
"type" : "int" , "default" : 0 }, { "name" : "intnum5", "type" : "int",
"default" : 0 }, { "name" : "intnum100", "type" : "int", "default" : 0 } ] } }
');
{code}
Opening the file shows the following schema
{noformat}
avro.schema
{"type":"record","name":"rmyrecord","fields":[{"name":"id","type":"int"},{"name":"intnum5","type":"int"},{"name":"intnum100","type":"int"}]}
{noformat}
There seems to be a problem storing the schema.
Viraj
was:
Script which stores Avro using a predefined schema does not store the default
values in the file
{code}
a = LOAD 'numbers.txt' USING PigStorage (':') as (intnum1000: int,id:
int,intnum5: int,intnum100: int,intnum: int,longnum: long,floatnum:
float,doublenum: double);
b2 = foreach a generate id, intnum5, intnum100;
c2 = filter b2 by 110 <= id and id < 120;
STORE c2 INTO '/tmp/TestAvroStorage/testDefaultValueWrite' USING
org.apache.pig.piggybank.storage.avro.AvroStorage (' { "debug" : 5, "schema" :
{ "name" : "rmyrecord", "type" : "record", "fields" : [ { "name" : "id",
"type" : "int" , "default" : 0 }, { "name" : "intnum5", "type" : "int",
"default" : 0 }, { "name" : "intnum100", "type" : "int", "default" : 0 } ] } }
');
{code}
Opening the file shows the following schema
{quote}
avro.schema
{"type":"record","name":"rmyrecord","fields":[{"name":"id","type":"int"},{"name":"intnum5","type":"int"},{"name":"intnum100","type":"int"}]}
{quote}
There seems to be a problem storing the schema.
Viraj
> Default values not stored in avro file when using specific schemas during
> store in AvroStorage
> ----------------------------------------------------------------------------------------------
>
> Key: PIG-3331
> URL: https://issues.apache.org/jira/browse/PIG-3331
> Project: Pig
> Issue Type: Bug
> Components: piggybank
> Affects Versions: 0.11.1
> Reporter: Viraj Bhat
> Assignee: Viraj Bhat
> Fix For: 0.11.2
>
>
> Script which stores Avro using a predefined schema does not store the default
> values in the file
> {code}
> a = LOAD 'numbers.txt' USING PigStorage (':') as (intnum1000: int,id:
> int,intnum5: int,intnum100: int,intnum: int,longnum: long,floatnum:
> float,doublenum: double);
> b2 = foreach a generate id, intnum5, intnum100;
> c2 = filter b2 by 110 <= id and id < 120;
> STORE c2 INTO '/tmp/TestAvroStorage/testDefaultValueWrite' USING
> org.apache.pig.piggybank.storage.avro.AvroStorage (' { "debug" : 5, "schema" :
> { "name" : "rmyrecord", "type" : "record", "fields" : [ { "name" : "id",
> "type" : "int" , "default" : 0 }, { "name" : "intnum5", "type" : "int",
> "default" : 0 }, { "name" : "intnum100", "type" : "int", "default" : 0 } ] } }
> ');
> {code}
> Opening the file shows the following schema
> {noformat}
> avro.schema
> {"type":"record","name":"rmyrecord","fields":[{"name":"id","type":"int"},{"name":"intnum5","type":"int"},{"name":"intnum100","type":"int"}]}
> {noformat}
> There seems to be a problem storing the schema.
> Viraj
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
