[jira] [Commented] (SPARK-34362) scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 1.4

2021-02-04 Thread Svitlana Ponomarova (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-34362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278954#comment-17278954
 ] 

Svitlana Ponomarova commented on SPARK-34362:
-

According to Google Dataproc 1.4 documentation it supports Apache Hive 2.3.7:
[https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]

As I see from sources in *branch-2.4*:
[https://github.com/apache/spark/blob/e7acca22cd1ed9a70fabc9ca143aa06fa8573864/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala#L101]

case "2.3" doesn't have any match to 2.3.7 version
which causes MatchError.

May this part be back-ported from *branch-3.0 ?*
https://github.com/apache/spark/blob/06942331a7db1e6d5e6709ac7009c180c94cc7c0/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala#L107

> scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 
> 1.4
> -
>
> Key: SPARK-34362
> URL: https://issues.apache.org/jira/browse/SPARK-34362
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.4.7
>Reporter: Svitlana Ponomarova
>Priority: Critical
>
> According to Google Dataproc 1.4 release notes:
>  
> [https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]
>  2.4.7 is supported spark version.
> Use *spark-hive_2.11-2.4.7.jar* for Hive on Dataproc 1.4 causes:
> {noformat}
> scala.MatchError: 2.3 (of class java.lang.String) scala.MatchError: 2.3 
> (of class java.lang.String) 
> at 
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.hiveVersion(IsolatedClientLoader.scala:89)
>  
> at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:300)
>  at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
>  at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
>  
> at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Comment Edited] (SPARK-34362) scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 1.4

2021-02-04 Thread Svitlana Ponomarova (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-34362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278938#comment-17278938
 ] 

Svitlana Ponomarova edited comment on SPARK-34362 at 2/4/21, 4:33 PM:
--

Maybe "Components" field is wrong. Unfortunately, I don't see "Hive" in a list. 


was (Author: sponomarova):
Maybe "Components" field was filled wrong. Unfortunately, I don't see "Hive" in 
a list. 

> scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 
> 1.4
> -
>
> Key: SPARK-34362
> URL: https://issues.apache.org/jira/browse/SPARK-34362
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.4.7
>Reporter: Svitlana Ponomarova
>Priority: Critical
>
> According to Google Dataproc 1.4 release notes:
>  
> [https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]
>  2.4.7 is supported spark version.
> Use *spark-hive_2.11-2.4.7.jar* for Hive on Dataproc 1.4 causes:
> {noformat}
> scala.MatchError: 2.3 (of class java.lang.String) scala.MatchError: 2.3 
> (of class java.lang.String) 
> at 
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.hiveVersion(IsolatedClientLoader.scala:89)
>  
> at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:300)
>  at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
>  at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
>  
> at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-34362) scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 1.4

2021-02-04 Thread Svitlana Ponomarova (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-34362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Svitlana Ponomarova updated SPARK-34362:

Description: 
According to Google Dataproc 1.4 release notes:
 
[https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]
 2.4.7 is supported spark version.

Use *spark-hive_2.11-2.4.7.jar* for Hive on Dataproc 1.4 causes:
{noformat}
scala.MatchError: 2.3 (of class java.lang.String) scala.MatchError: 2.3 
(of class java.lang.String) 
at 
org.apache.spark.sql.hive.client.IsolatedClientLoader$.hiveVersion(IsolatedClientLoader.scala:89)
 
at 
org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:300) 
at 
org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 
at 
org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
 
at 
org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
{noformat}

  was:
According to Google Dataproc 1.4 release notes:
 
[https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]
 2.4.7 is supported spark version.

Use spark-hive_2.11-2.4.7.jar for Hive on Dataproc 1.4 causes:
{noformat}
scala.MatchError: 2.3 (of class java.lang.String) scala.MatchError: 2.3 
(of class java.lang.String) 
at 
org.apache.spark.sql.hive.client.IsolatedClientLoader$.hiveVersion(IsolatedClientLoader.scala:89)
 
at 
org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:300) 
at 
org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 
at 
org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
 
at 
org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
{noformat}


> scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 
> 1.4
> -
>
> Key: SPARK-34362
> URL: https://issues.apache.org/jira/browse/SPARK-34362
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.4.7
>Reporter: Svitlana Ponomarova
>Priority: Critical
>
> According to Google Dataproc 1.4 release notes:
>  
> [https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]
>  2.4.7 is supported spark version.
> Use *spark-hive_2.11-2.4.7.jar* for Hive on Dataproc 1.4 causes:
> {noformat}
> scala.MatchError: 2.3 (of class java.lang.String) scala.MatchError: 2.3 
> (of class java.lang.String) 
> at 
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.hiveVersion(IsolatedClientLoader.scala:89)
>  
> at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:300)
>  at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
>  at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
>  
> at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-34362) scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 1.4

2021-02-04 Thread Svitlana Ponomarova (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-34362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278938#comment-17278938
 ] 

Svitlana Ponomarova commented on SPARK-34362:
-

Maybe "Components" field was filled wrong. Unfortunately, I don't see "Hive" in 
a list. 

> scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 
> 1.4
> -
>
> Key: SPARK-34362
> URL: https://issues.apache.org/jira/browse/SPARK-34362
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.4.7
>Reporter: Svitlana Ponomarova
>Priority: Critical
>
> According to Google Dataproc 1.4 release notes:
>  
> [https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]
>  2.4.7 is supported spark version.
> Use spark-hive_2.11-2.4.7.jar for Hive on Dataproc 1.4 causes:
> {noformat}
> scala.MatchError: 2.3 (of class java.lang.String) scala.MatchError: 2.3 
> (of class java.lang.String) 
> at 
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.hiveVersion(IsolatedClientLoader.scala:89)
>  
> at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:300)
>  at 
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
>  at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
>  
> at 
> org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-34362) scala.MatchError: 2.3 (of class java.lang.String) for Hive on Google Dataproc 1.4

2021-02-04 Thread Svitlana Ponomarova (Jira) 
Svitlana Ponomarova created SPARK-34362:
---

 Summary: scala.MatchError: 2.3 (of class java.lang.String) for 
Hive on Google Dataproc 1.4
 Key: SPARK-34362
 URL: https://issues.apache.org/jira/browse/SPARK-34362
 Project: Spark
  Issue Type: Bug
  Components: SQL
Affects Versions: 2.4.7
Reporter: Svitlana Ponomarova


According to Google Dataproc 1.4 release notes:
 
[https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4]
 2.4.7 is supported spark version.

Use spark-hive_2.11-2.4.7.jar for Hive on Dataproc 1.4 causes:
{noformat}
scala.MatchError: 2.3 (of class java.lang.String) scala.MatchError: 2.3 
(of class java.lang.String) 
at 
org.apache.spark.sql.hive.client.IsolatedClientLoader$.hiveVersion(IsolatedClientLoader.scala:89)
 
at 
org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:300) 
at 
org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 
at 
org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
 
at 
org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org