[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039904#comment-17039904 ]
Ismaël Mejía edited comment on SPARK-26346 at 2/19/20 10:54 AM: ---------------------------------------------------------------- It seems this issue is blocked because Spark is having dependency issues because of its dependency on Hive. Hive depends on both Avro 1.7 (for Hive 1.x) and Avro 1.8 (for Hive 2.x). Parquet depends on Avro 1.9. So there are issues from API changes on Avro. I proposed a patch for Hive with the hope that it gets backported into Hive 2.x to unlock the Avro 1.9 upgrade on Spark (and transitively this issue) but so far I have not received any attention from the Hive community, for more info HIVE-21737 and [my multiples call for attention in the Hive ML|[https://lists.apache.org/thread.html/rc6c672ad4a5e255957d54d80ff83bf48eacece2828a86bc6cedd9c4c%40%3Cdev.hive.apache.org%3E].] If someone can ping somebody in the Hive community that can help this advance that would be great! was (Author: iemejia): It seems this issue is blocked because Spark is having dependency issues because of its dependency on Hive. Hive depends on both Avro 1.7 (for Hive 1.x) and Avro 1.8 (for Hive 2.x). Parquet depends on Avro 1.9. So there are issues from API changes on Avro. I proposed a patch for Hive with the hope that it gets backported into Hive 2.x but so far I have not received any attention from the Hive community, for more info HIVE-21737 and [my multiples call for attention in the Hive ML|[https://lists.apache.org/thread.html/rc6c672ad4a5e255957d54d80ff83bf48eacece2828a86bc6cedd9c4c%40%3Cdev.hive.apache.org%3E].] If someone can ping somebody in the Hive community that can help this advance that would be great! > Upgrade parquet to 1.11.1 > ------------------------- > > Key: SPARK-26346 > URL: https://issues.apache.org/jira/browse/SPARK-26346 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 3.0.0 > Reporter: Yuming Wang > Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org