[
https://issues.apache.org/jira/browse/SPARK-61067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61067:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Keep the tags of a session per thread in the Python client as PySpark does
> --------------------------------------------------------------------------
>
> Key: SPARK-61067
> URL: https://issues.apache.org/jira/browse/SPARK-61067
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> In PySpark Connect, the tags of {{spark.addTag}}, {{removeTag}}, {{getTags}}
> and {{clearTags}} are those of the current thread: the operations that a
> thread starts carry its tags only, so {{spark.interruptTag(tag)}} interrupts
> only those of the threads that added the tag, and a new thread starts without
> tags.
> The Rust-backed Python client keeps one set of tags for all the threads of a
> session: a tag that one thread adds is on the operations of every thread, so
> {{interruptTag}} interrupts them all, and {{getTags}} of a new thread returns
> the tags of the others. Keep them per thread as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
