Xiangrui Meng created SPARK-2755:
------------------------------------
Summary: TorrentBroadcast cannot broadcast very large objects
Key: SPARK-2755
URL: https://issues.apache.org/jira/browse/SPARK-2755
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.0.0
Reporter: Xiangrui Meng
TorrentBroadcast uses `Utils.serialize` to serialize an object into
Array[Byte]. So it cannot handle data of size greater than Int.MaxValue bytes.
Instead of serializing the object into Array[Byte] directly, we can use the
stream version implemented in HttpBroadcast.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
