[
https://issues.apache.org/jira/browse/HIVE-7049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997378#comment-13997378
]
Mohammad Kamrul Islam commented on HIVE-7049:
---------------------------------------------
Thank [~xuefuz] for the comments.
I believe it is a valid Avro schema evolution.
Please see the following comments copied from the link:
http://avro.apache.org/docs/1.7.6/spec.html#Schema+Resolution
{noformat}
* if reader's is a union, but writer's is not
The first schema in the reader's union that matches the writer's schema is
recursively resolved against it. If none match, an error is signalled.
* if writer's is a union, but reader's is not
If the reader's schema matches the selected writer's schema, it is recursively
resolved against it. If they do not match, an error is signalled.
{noformat}
Moreover, i tested a similar scenarios using pure avro code where i wrote using
schema "string" and read it using ["null","string"].
> Unable to deserialize AVRO data when file schema and record schema are
> different and nullable
> ---------------------------------------------------------------------------------------------
>
> Key: HIVE-7049
> URL: https://issues.apache.org/jira/browse/HIVE-7049
> Project: Hive
> Issue Type: Bug
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
> Attachments: HIVE-7049.1.patch
>
>
> It mainly happens when
> 1 )file schema and record schema are not same
> 2 ) Record schema is nullable but file schema is not.
> The potential code location is at class AvroDeserialize
>
> {noformat}
> if(AvroSerdeUtils.isNullableType(recordSchema)) {
> return deserializeNullableUnion(datum, fileSchema, recordSchema,
> columnType);
> }
> {noformat}
> In the above code snippet, recordSchema is verified if it is nullable. But
> the file schema is not checked.
> I tested with these values:
> {noformat}
> recordSchema= ["null","string"]
> fielSchema= "string"
> {noformat}
> And i got the following exception <line numbers might not be the same due to
> mu debugged code version>.
> {noformat}
> org.apache.avro.AvroRuntimeException: Not a union: "string"
> at org.apache.avro.Schema.getTypes(Schema.java:272)
> at
> org.apache.hadoop.hive.serde2.avro.AvroDeserializer.deserializeNullableUnion(AvroDeserializer.java:275)
> at
> org.apache.hadoop.hive.serde2.avro.AvroDeserializer.worker(AvroDeserializer.java:205)
> at
> org.apache.hadoop.hive.serde2.avro.AvroDeserializer.workerBase(AvroDeserializer.java:188)
> at
> org.apache.hadoop.hive.serde2.avro.AvroDeserializer.deserialize(AvroDeserializer.java:174)
> at
> org.apache.hadoop.hive.serde2.avro.TestAvroDeserializer.verifyNullableType(TestAvroDeserializer.java:487)
> at
> org.apache.hadoop.hive.serde2.avro.TestAvroDeserializer.canDeserializeNullableTypes(TestAvroDeserializer.java:407)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
