[
https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270771#comment-14270771
]
Reynold Xin commented on SPARK-5124:
------------------------------------
Thanks for doing this. I haven't looked through all the changes yet, but here
are some quick comments:
1. For DAGScheduler, we are probably OK to just use an event loop instead of an
actor. Just put some messages into a queue, and have a loop that processes the
queue. Otherwise we are making every call to DAGScheduler going through a
socket and that can severely impact scheduler throughput. (Although I haven't
looked closely at your change so maybe you are doing a different thing here)
2. Is it really that expensive to listen to network events that warrant a
separate NetworkRpcEndpoint?
> Standardize internal RPC interface
> ----------------------------------
>
> Key: SPARK-5124
> URL: https://issues.apache.org/jira/browse/SPARK-5124
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Reynold Xin
> Assignee: Shixiong Zhu
> Attachments: Pluggable RPC - draft 1.pdf
>
>
> In Spark we use Akka as the RPC layer. It would be great if we can
> standardize the internal RPC interface to facilitate testing. This will also
> provide the foundation to try other RPC implementations in the future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
