[jira] [Assigned] (SPARK-33376) Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
[
https://issues.apache.org/jira/browse/SPARK-33376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-33376:
Assignee: Chao Sun (was: Apache Spark)
> Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
> ---
>
> Key: SPARK-33376
> URL: https://issues.apache.org/jira/browse/SPARK-33376
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 3.0.1
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Fix For: 3.1.0
>
>
> Currently, when initializing {{IsolatedClientLoader}}, ppl can specify to
> either share Hadoop classes from Spark or not. In the latter case it's
> supposed to only loads the Hadoop classes from the Hive jars themselves.
> However this feature is currently used in two cases: 1) unit tests, 2) when
> the Hadoop version defined in Maven can not be found when
> {{spark.sql.hive.metastore.jars == "maven"}}. Also when
> {{sharesHadoopClasses}} is false, it isn't really only using Hadoop classes
> from Hive jars: Spark also download {{hadoop-client}} jar and put it together
> with the Hive jars, and the Hadoop version used by {{hadoop-client}} is the
> same version used by Spark itself. This could potentially cause issues
> because we are mixing two versions of Hadoop jars in the classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-33376) Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
[
https://issues.apache.org/jira/browse/SPARK-33376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33376:
Assignee: (was: Apache Spark)
> Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
> ---
>
> Key: SPARK-33376
> URL: https://issues.apache.org/jira/browse/SPARK-33376
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 3.0.1
>Reporter: Chao Sun
>Priority: Major
>
> Currently, when initializing {{IsolatedClientLoader}}, ppl can specify to
> either share Hadoop classes from Spark or not. In the latter case it's
> supposed to only loads the Hadoop classes from the Hive jars themselves.
> However this feature is currently used in two cases: 1) unit tests, 2) when
> the Hadoop version defined in Maven can not be found when
> {{spark.sql.hive.metastore.jars == "maven"}}. Also when
> {{sharesHadoopClasses}} is false, it isn't really only using Hadoop classes
> from Hive jars: Spark also download {{hadoop-client}} jar and put it together
> with the Hive jars, and the Hadoop version used by {{hadoop-client}} is the
> same version used by Spark itself. This could potentially cause issues
> because we are mixing two versions of Hadoop jars in the classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-33376) Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
[
https://issues.apache.org/jira/browse/SPARK-33376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33376:
Assignee: Apache Spark
> Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
> ---
>
> Key: SPARK-33376
> URL: https://issues.apache.org/jira/browse/SPARK-33376
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 3.0.1
>Reporter: Chao Sun
>Assignee: Apache Spark
>Priority: Major
>
> Currently, when initializing {{IsolatedClientLoader}}, ppl can specify to
> either share Hadoop classes from Spark or not. In the latter case it's
> supposed to only loads the Hadoop classes from the Hive jars themselves.
> However this feature is currently used in two cases: 1) unit tests, 2) when
> the Hadoop version defined in Maven can not be found when
> {{spark.sql.hive.metastore.jars == "maven"}}. Also when
> {{sharesHadoopClasses}} is false, it isn't really only using Hadoop classes
> from Hive jars: Spark also download {{hadoop-client}} jar and put it together
> with the Hive jars, and the Hadoop version used by {{hadoop-client}} is the
> same version used by Spark itself. This could potentially cause issues
> because we are mixing two versions of Hadoop jars in the classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-33376) Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
[
https://issues.apache.org/jira/browse/SPARK-33376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33376:
Assignee: Apache Spark
> Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader
> ---
>
> Key: SPARK-33376
> URL: https://issues.apache.org/jira/browse/SPARK-33376
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 3.0.1
>Reporter: Chao Sun
>Assignee: Apache Spark
>Priority: Major
>
> Currently, when initializing {{IsolatedClientLoader}}, ppl can specify to
> either share Hadoop classes from Spark or not. In the latter case it's
> supposed to only loads the Hadoop classes from the Hive jars themselves.
> However this feature is currently used in two cases: 1) unit tests, 2) when
> the Hadoop version defined in Maven can not be found when
> {{spark.sql.hive.metastore.jars == "maven"}}. Also when
> {{sharesHadoopClasses}} is false, it isn't really only using Hadoop classes
> from Hive jars: Spark also download {{hadoop-client}} jar and put it together
> with the Hive jars, and the Hadoop version used by {{hadoop-client}} is the
> same version used by Spark itself. This could potentially cause issues
> because we are mixing two versions of Hadoop jars in the classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
