HyukjinKwon commented on PR #59277:
URL: https://github.com/apache/spark/pull/59277#issuecomment-6135973587
Thanks @venkata91, I checked 42cc9b6fbd. The conf doc and the
`docs/configuration.md` row now both say not to stop or release the old
`SparkSession` once another client may have created a new session with the same
id, because `ReleaseSession` doesn't check the server-side session id. A
follow-up JIRA for adding `client_observed_server_side_session_id` to
`ReleaseSessionRequest` sounds good to me. One thing for that follow-up: the
Scala client's `SparkSession.close()` still calls `releaseSession()` after the
client has seen `SESSION_CHANGED`, because it doesn't check
`client.isSessionValid`. Python's `stop()` already skips the release once the
client is closed, so a matching check in Scala's `close()` would cover an old
handle that has already seen `SESSION_CHANGED`, even before the proto change.
<!-- SPARK_DEV_REVIEW_REPLY
{"feedback_id":"conversation:6131717412","thread_id":"conversation:6131717412","verdict_sha256":"11fab2560ffcbd15365598e58ce703dc6321278a87cc8b2a49c8edabe3bda4c3"}
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
