j1wonpark commented on code in PR #57653:
URL: https://github.com/apache/spark/pull/57653#discussion_r3695697727
##########
python/pyspark/sql/tests/streaming/test_streaming_foreach_batch.py:
##########
@@ -119,6 +119,24 @@ def func(df: DataFrame, batch_id: int):
)
self.assertEqual(sorted(df.collect()), sorted(actual.collect()))
+ def test_streaming_foreach_batch_operation_tags(self):
Review Comment:
Although the fix is specific to the Spark Connect `foreachBatch` worker, the
same test also passes in Classic through its existing session-cloning path. I
kept it in the shared mixin to verify that both implementations preserve tags
inside `foreachBatch` callbacks. Would you prefer moving it to the Connect-only
suite to keep the regression test scoped to the changed implementation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
