[
https://issues.apache.org/jira/browse/SPARK-49458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Changgyoo Park updated SPARK-49458:
-----------------------------------
Description:
Related to SPARK-47380.
Driver restart -> OPERATION_NOT_FOUND instead of SESSION_NOT_FOUND because the
first request of a certain message type is always verified as legit in the
current implementation.
E.g.,
Start execution (client_observed_server_side_session_id is set) -> driver
restart -> reattach (client_observed_server_side_session_id is not set) ->
OPERATION_NOT_FOUND instead of SESSION_NOT_FOUND.
Remaining question is, once the driver is gone, all the session-local data is
gone, including temp tables, UDFs, cached relations, but the server-side
session id is only stored per request. In the long run, the server-side session
id must be stored in the client object.
* Lifetime: current = request -> ideally = session.
was:
Related to SPARK-47380.
Driver restart -> OPERATION_NOT_FOUND instead of SESSION_NOT_FOUND because the
first request of a certain message type is always verified as legit in the
current implementation.
E.g.,
Start execution (client_observed_server_side_session_id is set) -> driver
restart -> reattach (client_observed_server_side_session_id is not set) ->
OPERATION_NOT_FOUND instead of SESSION_NOT_FOUND.
> ReattachExecute server-side session id validation does not catch a driver
> restart
> ---------------------------------------------------------------------------------
>
> Key: SPARK-49458
> URL: https://issues.apache.org/jira/browse/SPARK-49458
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 4.0.0
> Reporter: Changgyoo Park
> Priority: Major
>
> Related to SPARK-47380.
> Driver restart -> OPERATION_NOT_FOUND instead of SESSION_NOT_FOUND because
> the first request of a certain message type is always verified as legit in
> the current implementation.
> E.g.,
> Start execution (client_observed_server_side_session_id is set) -> driver
> restart -> reattach (client_observed_server_side_session_id is not set) ->
> OPERATION_NOT_FOUND instead of SESSION_NOT_FOUND.
> Remaining question is, once the driver is gone, all the session-local data is
> gone, including temp tables, UDFs, cached relations, but the server-side
> session id is only stored per request. In the long run, the server-side
> session id must be stored in the client object.
> * Lifetime: current = request -> ideally = session.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
