Alex Angelini created SPARK-4603: ------------------------------------ Summary: EOF when broadcasting a dict with an empty string value. Key: SPARK-4603 URL: https://issues.apache.org/jira/browse/SPARK-4603 Project: Spark Issue Type: Bug Components: PySpark Affects Versions: 1.1.0 Environment: OSX 10.10 Reporter: Alex Angelini
Steps to reproduce: 1. Broadcast {'a': ''} 2. Try to read the value of the broadcast {code} Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /__ / .__/\_,_/_/ /_/\_\ version 1.3.0-SNAPSHOT /_/ Using Python version 2.7.8 (default, Oct 19 2014 16:02:00) SparkContext available as sc. In [1]: sc Out[1]: <pyspark.context.SparkContext at 0x1077e4710> In [2]: b = sc.broadcast({'a': ''}) In [3]: b.value --------------------------------------------------------------------------- EOFError Traceback (most recent call last) <ipython-input-3-59996048613f> in <module>() ----> 1 b.value /Users/alexangelini/src/starscream/spark/current/python/pyspark/broadcast.pyc in value(self) 75 if not hasattr(self, "_value") and self.path is not None: 76 ser = LargeObjectSerializer() ---> 77 self._value = ser.load_stream(open(self.path)).next() 78 return self._value 79 /Users/alexangelini/src/starscream/spark/current/python/pyspark/serializers.pyc in load_stream(self, stream) 615 yield value 616 elif type == 'P': --> 617 yield cPickle.load(stream) 618 else: 619 raise ValueError("unknown type: %s" % type) EOFError: {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org