[
https://issues.apache.org/jira/browse/PIG-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Bhat updated PIG-3318:
----------------------------
Attachment: PIG-3318_2.patch
> AVRO: 'default value' not honored when merging schemas on load with
> AvroStorage
> -------------------------------------------------------------------------------
>
> Key: PIG-3318
> URL: https://issues.apache.org/jira/browse/PIG-3318
> Project: Pig
> Issue Type: Bug
> Components: piggybank
> Affects Versions: 0.11.2
> Reporter: Viraj Bhat
> Assignee: Viraj Bhat
> Labels: patch
> Fix For: 0.12, 0.11.2
>
> Attachments: Employee3.ser, Employee4.ser, Employee6.ser,
> expected_testMultipleSchemasWithDefaultValue.avro, PIG-3318_2.patch
>
>
> Piggybank - AvroStorage. When merging multiple schemas where default values
> have been specified in the avro schema;
> The AvroStorage puts nulls in the merged data set.
> ==> Employee3.avro <==
> {
> "type" : "record",
> "name" : "employee",
> "fields":[
> {"name" : "name", "type" : "string", "default" : "NU"},
> {"name" : "age", "type" : "int", "default" : 0 },
> {"name" : "dept", "type": "string", "default" : "DU"} ] }
> ==> Employee4.avro <==
> {
> "type" : "record",
> "name" : "employee",
> "fields":[
> {"name" : "name", "type" : "string", "default" : "NU"},
> {"name" : "age", "type" : "int", "default" : 0},
> {"name" : "dept", "type": "string", "default" : "DU"},
> {"name" : "office", "type": "string", "default" : "OU"} ] }
> ==> Employee6.avro <==
> {
> "type" : "record",
> "name" : "employee",
> "fields":[
> {"name" : "name", "type" : "string", "default" : "NU"},
> {"name" : "lastname", "type": "string", "default" : "LNU"},
> {"name" : "age", "type" : "int","default" : 0},
> {"name" : "salary", "type": "int", "default" : 0},
> {"name" : "dept", "type": "string","default" : "DU"},
> {"name" : "office", "type": "string","default" : "OU"} ] }
> The pig script:
> employee = load 'employee{3,4,6}.ser' using
> org.apache.pig.piggybank.storage.avro.AvroStorage('multiple_schemas');
> describe employee;
> dump employee;
> Output Schema:
> employee: {name: chararray,age: int,dept: chararray,lastname:
> chararray,salary: int,office: chararray}
> (Milo,30,DH,,,)
> (Asmya,34,PQ,,,)
> (Baljit,23,RS,,,)
> (Pune,60,Astrophysics,Warriors,5466,UTA)
> (Rajsathan,20,Biochemistry,Royals,1378,Stanford)
> (Chennai,50,Microbiology,Superkings,7338,Hopkins)
> (Mumbai,20,Applied Math,Indians,4468,UAH)
> (Praj,54,RMX,,,Champaign)
> (Buba,767,HD,,,Sunnyvale)
> (Manku,375,MS,,,New York)
> Regards
> Viraj
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
