[
https://issues.apache.org/jira/browse/SPARK-60769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qianlan Chen updated SPARK-60769:
---------------------------------
Affects Version/s: 4.4.0
(was: 5.0.0)
> Read Maven resolution settings from session configuration
> ---------------------------------------------------------
>
> Key: SPARK-60769
> URL: https://issues.apache.org/jira/browse/SPARK-60769
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: Qianlan Chen
> Priority: Major
>
> SparkSession.addArtifact is a session-level API, but its Maven resolver reads
> the Ivy settings file, local repository path, and connection/read timeouts
> directly from SparkContext's SparkConf. Consequently, it ignores
> session-specific Maven configuration.
> Read spark.jars.ivySettings, spark.jars.ivy, spark.jars.ivyConnectTimeout,
> and spark.jars.ivyReadTimeout from the owning SparkSession configuration.
> Preserve variable substitution, timeout defaults and validation, the optional
> local repository path, and session-local runtime caches. Cluster-configured
> values continue to be inherited. Existing public configuration-write rules
> remain in place.
> Update the configuration documentation and add regression coverage for
> inherited configuration, session-specific settings and local repository
> resolution, timeout parsing/defaults/validation, and session isolation.
> Pull request: https://github.com/apache/spark/pull/59977
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
