Hi,all:
aused by: org.apache.spark.api.python.PythonException: Traceback (most
recent call last):
File "/home/ubuntu/spark-2.4.3/python/lib/pyspark.zip/pyspark/worker.py",
line 364, in main
func, profiler, deserializer, serializer = read_command(pickleSer,
infile)
File "/home/ubuntu/spark-2.4.3/python/lib/pyspark.zip/pyspark/worker.py",
line 69, in read_command
command = serializer._read_with_length(file)
File
"/home/ubuntu/spark-2.4.3/python/lib/pyspark.zip/pyspark/serializers.py",
line 172, in _read_with_length
return self.loads(obj)
File
"/home/ubuntu/spark-2.4.3/python/lib/pyspark.zip/pyspark/serializers.py",
line 583, in loads
return pickle.loads(obj)
ImportError: No module named
risk_platform.mutli_process_thread.decision_tree
Is the problem caused by comunication between spark master and slave in
pickle way?
Python cannot pickle instancemethod and code object,So that is why load
error?
Thanks for answer!
