Reynold Xin created SPARK-3132:
----------------------------------
Summary: Avoid serialization for Array[Byte] in TorrentBroadcast
Key: SPARK-3132
URL: https://issues.apache.org/jira/browse/SPARK-3132
Project: Spark
Issue Type: Sub-task
Reporter: Reynold Xin
If the input data is a byte array, we should allow TorrentBroadcast to skip
serializing and compressing the input.
To do this, we should add a new parameter (shortCircuitByteArray) to
TorrentBroadcast, and then avoid serialization in if the input is byte array
and shortCircuitByteArray is true.
We should then also do compression in task serialization itself instead of
doing it in TorrentBroadcast.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
