WweiL commented on code in PR #46125:
URL: https://github.com/apache/spark/pull/46125#discussion_r1571451873
##########
python/pyspark/sql/tests/connect/streaming/test_parity_foreach_batch.py:
##########
@@ -66,6 +66,26 @@ def func(df, _):
q = df.writeStream.foreachBatch(func).start()
q.processAllAvailable()
+ def test_pickling_deserialization_error(self):
+ class NoUnpickle:
+ def __reduce__(self):
+ if isinstance(self, type(None)):
+ raise TypeError("Cannot unpickle instance of NoUnpickle")
+ return type(self), ()
Review Comment:
I'm having a hard time parsing this code
so what's happening here is, during unpickle, `type(self)`, which is defined
in `__repr__` is called, then somehow "self" is None...? And the TypeError is
raised?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
