[
https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043738#comment-14043738
]
Patrick Wendell commented on SPARK-1112:
----------------------------------------
[~reachbach] If you are running on standalone mode, it might work if you go on
every node in your cluster and add the following to spark-env.sh:
{code}
SPARK_JAVA_OPTS="-Dspark.akka.frameSize=XXX"
{code}
However, this work around will only work if every job in your cluster is using
the same frame size (XXX).
The main recommendation is to upgrade to 1.0.1. We are very conservative about
what we merge into maintenance branches, so we recommend users upgrade
immediately once we release them.
> When spark.akka.frameSize > 10, task results bigger than 10MiB block execution
> ------------------------------------------------------------------------------
>
> Key: SPARK-1112
> URL: https://issues.apache.org/jira/browse/SPARK-1112
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 0.9.0, 1.0.0
> Reporter: Guillaume Pitel
> Assignee: Xiangrui Meng
> Priority: Blocker
> Fix For: 1.0.1, 1.1.0
>
>
> When I set the spark.akka.frameSize to something over 10, the messages sent
> from the executors to the driver completely block the execution if the
> message is bigger than 10MiB and smaller than the frameSize (if it's above
> the frameSize, it's ok)
> Workaround is to set the spark.akka.frameSize to 10. In this case, since
> 0.8.1, the blockManager deal with the data to be sent. It seems slower than
> akka direct message though.
> The configuration seems to be correctly read (see actorSystemConfig.txt), so
> I don't see where the 10MiB could come from
--
This message was sent by Atlassian JIRA
(v6.2#6252)
