WweiL commented on code in PR #46125: URL: https://github.com/apache/spark/pull/46125#discussion_r1571451873
########## python/pyspark/sql/tests/connect/streaming/test_parity_foreach_batch.py: ########## @@ -66,6 +66,26 @@ def func(df, _): q = df.writeStream.foreachBatch(func).start() q.processAllAvailable() + def test_pickling_deserialization_error(self): + class NoUnpickle: + def __reduce__(self): + if isinstance(self, type(None)): + raise TypeError("Cannot unpickle instance of NoUnpickle") + return type(self), () Review Comment: I'm having a hard time parsing this code so what's happening here is, during unpickle, `type(self)`, which is defined in `__repr__` is called, then somehow "self" is None...? And the TypeError is raised? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org