[ 
https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039904#comment-17039904
 ] 

Ismaël Mejía edited comment on SPARK-26346 at 2/19/20 10:54 AM:
----------------------------------------------------------------

It seems this issue is blocked because Spark is having dependency issues 
because of its dependency on Hive. Hive depends on both Avro 1.7 (for Hive 1.x) 
and Avro 1.8 (for Hive 2.x). Parquet depends on Avro 1.9. So there are issues 
from API changes on Avro. I proposed a patch for Hive with the hope that it 
gets backported into Hive 2.x to unlock the Avro 1.9 upgrade on Spark (and 
transitively this issue) but so far I have not received any attention from the 
Hive community, for more info HIVE-21737 and [my multiples call for attention 
in the Hive 
ML|[https://lists.apache.org/thread.html/rc6c672ad4a5e255957d54d80ff83bf48eacece2828a86bc6cedd9c4c%40%3Cdev.hive.apache.org%3E].]
 If someone can ping somebody in the Hive community that can help this advance 
that would be great!


was (Author: iemejia):
It seems this issue is blocked because Spark is having dependency issues 
because of its dependency on Hive. Hive depends on both Avro 1.7 (for Hive 1.x) 
and Avro 1.8 (for Hive 2.x). Parquet depends on Avro 1.9. So there are issues 
from API changes on Avro. I proposed a patch for Hive with the hope that it 
gets backported into Hive 2.x but so far I have not received any attention from 
the Hive community, for more info HIVE-21737 and [my multiples call for 
attention in the Hive 
ML|[https://lists.apache.org/thread.html/rc6c672ad4a5e255957d54d80ff83bf48eacece2828a86bc6cedd9c4c%40%3Cdev.hive.apache.org%3E].]
 If someone can ping somebody in the Hive community that can help this advance 
that would be great!

> Upgrade parquet to 1.11.1
> -------------------------
>
>                 Key: SPARK-26346
>                 URL: https://issues.apache.org/jira/browse/SPARK-26346
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Yuming Wang
>            Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to