qianlan717 opened a new pull request, #59982: URL: https://github.com/apache/spark/pull/59982
### What changes were proposed in this pull request? This PR adds the public session configuration `spark.connect.session.serverSideMavenResolution.enabled`, defaulting to `false`. The Spark Connect JVM client sends an eligible `ivy://` dependency to the server only when the server advertises support, the session configuration is enabled, and the URI does not specify repositories through `repos`. Otherwise, the dependency is resolved by the client. The server capability remains cached support metadata, while the session preference is read for every `addArtifact` or `addArtifacts` call so it can change at runtime. The configuration is a JVM-client routing preference, not a server-side enforcement control. The server continues to accept Maven dependency requests sent directly by protocol clients. This is split from #59977, which makes the client and server resolvers honor session-level Maven settings. The PRs overlap in client routing tests and documentation; #59977 can be rebased to keep only the resolver-settings changes after this change lands. ### Why are the changes needed? JIRA: https://issues.apache.org/jira/browse/SPARK-60773 Server capability advertisement describes whether server-side Maven resolution is supported, but it should not select that path for every session. Separating cached capability detection from the mutable session preference keeps client-side resolution as the default and lets a session opt in when server-side repository access is desired. ### Does this PR introduce _any_ user-facing change? Yes, relative to the unreleased `master` behavior. An eligible Spark Connect JVM `ivy://` dependency is now resolved on the client by default instead of automatically using the server when the server advertises support. A session can opt in with: ```scala spark.conf.set( "spark.connect.session.serverSideMavenResolution.enabled", "true") ``` Dependencies with an explicit `repos` query parameter remain client-resolved. Spark Classic is unaffected. ### How was this patch tested? Added `ArtifactSuite` coverage for the default client-side path, the enabled server-side path, runtime session-configuration changes, explicit repositories, and servers without the capability. Ran: ```text build/sbt 'connect/Test/compile' \ 'connect-client-jvm/testOnly org.apache.spark.sql.connect.client.ArtifactSuite' ``` `ArtifactSuite`: 18 tests passed. `git diff --check`, changed-source line-length checks, and non-ASCII checks also passed. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
