[ https://issues.apache.org/jira/browse/HIVE-24797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287446#comment-17287446 ]
Yuming Wang commented on HIVE-24797: ------------------------------------ [~SteveNiemitz] Could you help verify other incompatible changes: [https://issues.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20resolution%20in%20(Fixed)%20AND%20cf%5B12310191%5D%20%3D%20%22Incompatible%20change%22%20AND%20fixVersion%20in%20(1.9.0%2C%201.9.1%2C%201.10.0%2C%201.10.1)%20%20%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC] > Disable validate default values when parsing Avro schemas > --------------------------------------------------------- > > Key: HIVE-24797 > URL: https://issues.apache.org/jira/browse/HIVE-24797 > Project: Hive > Issue Type: Bug > Reporter: Yuming Wang > Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > It will throw exceptions when upgrading Avro to 1.10.1 for this schema: > {code:json} > { > "type": "record", > "name": "EventData", > "doc": "event data", > "fields": [ > {"name": "ARRAY_WITH_DEFAULT", "type": {"type": "array", "items": > "string"}, "default": null } > ] > } > {code} > {noformat} > org.apache.avro.AvroTypeException: Invalid default for field > ARRAY_WITH_DEFAULT: null not a {"type":"array","items":"string"} > at org.apache.avro.Schema.validateDefault(Schema.java:1571) > at org.apache.avro.Schema.access$500(Schema.java:87) > at org.apache.avro.Schema$Field.<init>(Schema.java:544) > at org.apache.avro.Schema.parse(Schema.java:1678) > at org.apache.avro.Schema$Parser.parse(Schema.java:1425) > at org.apache.avro.Schema$Parser.parse(Schema.java:1396) > at > org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFor(AvroSerdeUtils.java:287) > at > org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFromFS(AvroSerdeUtils.java:170) > at > org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:139) > at > org.apache.hadoop.hive.serde2.avro.AvroSerDe.determineSchemaOrReturnErrorSchema(AvroSerDe.java:187) > at > org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:107) > at > org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:83) > at > org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:493) > at > org.apache.hadoop.hive.ql.metadata.Partition.getDeserializer(Partition.java:225) > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)