Sebastian Straub created SPARK-30270: ----------------------------------------
Summary: Can't pickle abstract classes (with cloudpickle) Key: SPARK-30270 URL: https://issues.apache.org/jira/browse/SPARK-30270 Project: Spark Issue Type: Bug Components: PySpark Affects Versions: 2.4.4 Reporter: Sebastian Straub I can't use any classes that are derived from abstract classes in PySpark, because cloudpickle can't pickle them. Example: {code:java} import pickle from abc import ABC from pyspark import cloudpickle class Foo(ABC): pass class Bar(Foo): pass bar = Bar() # pickle dump works fine pickle.dumps(bar) # cloudpickle doesn't cloudpickle.dumps(bar) {code} A similar bug has already been reported in SPARK-21439 and marked resolved, but I can confirm that the issue still persists. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org