HyukjinKwon opened a new pull request, #269: URL: https://github.com/apache/spark-connect-rust/pull/269
### What changes were proposed in this pull request? [SPARK-61067](https://issues.apache.org/jira/browse/SPARK-61067): the tags of a session are now those of each thread, as PySpark Connect keeps them. As PySpark's, `spark.addTag`, `removeTag`, `getTags` and `clearTags` are of the current thread: - the operations that a thread starts carry its tags only, so `spark.interruptTag(tag)` interrupts only those of the threads that added the tag - a new thread starts without tags - `getTags` returns a set The core `SparkSession` keeps the tags of each thread, and the requests that a thread builds carry them. ### Why are the changes needed? The client kept one set of tags for all the threads of a session: - a tag that one thread added was on the operations of every thread, so `interruptTag` interrupted them all - `getTags` of a new thread returned the tags of the others, as a list ### Does this PR introduce _any_ user-facing change? Yes. The tags are those of each thread, as in PySpark 4.3.0. ### How was this patch tested? - A new live test runs the slow queries of a tagged and an untagged thread, interrupts the tag, and checks that only the query of the tagged thread is interrupted. It passes with official PySpark 4.3.0, and fails without this change. - A new offline test checks the tags of a new thread, and the tags that the requests of each thread send to a mock Connect server. - A new lib test checks the tags of a thread of the core session. - The lib tests of both crates (584 passed), the build of all tests, also with `wasm-udf`, the offline Python suites (1930 passed, 8 skipped), and the live drop-in and Arrow suites (127 passed) and live cases (10 passed) against a local Spark 4.3.0 server. Clippy reports no diagnostics in the changed lines. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
