[
https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14335252#comment-14335252
]
Marcelo Vanzin edited comment on SPARK-5124 at 2/24/15 7:03 PM:
----------------------------------------------------------------
Hi @rxin,
Yeah, the current {{receive()}} gives you the message and the sender, but it
doesn't give you a way to explicitly reply to a message. That means two things:
- It's not clear whether calling something like {{sender.send()}} would be
thread-safe. For example, in akka it would because akka processes one message
at a time for each actor, so you're sending replies in order. But that forces
you to implement the same semantics on any new RPC layer, and that is one akka
feature thing that I'd prefer not to inherit (as you say, let each endpoint
decide whether it needs it).
- It precludes anynchronously handling that message for much of the same
reasons. e.g., let's say some incoming message requires a long computation
before sending the reply, and you use a thread pool for that. With
{{sender.send()}}, you'd need to build correlation logic into your application.
If you have something like {{message.reply()}} instead, you don't need to - the
RPC layer handles correlations and timing out if a reply doesn't arrive.
Granted, I have no idea how you'd implement that second bullet in akka, and
perhaps for that reason I don't think Spark currently uses that pattern
anywhere. But it sounds like a useful feature to have.
was (Author: vanzin):
Hi @rxin,
Yeah, the current {{receive()}} gives you the message and the sender, but it
doesn't give you a way to explicit reply to a message. That means two things:
- It's not clear whether calling something like {{sender.send()}} would be
thread-safe. For example, in akka it would because akka processes one message
at a time for each actor, so you're sending replies in order. But that forces
you to implement the same semantics on any new RPC layer, and that is one akka
feature thing that I'd prefer not to inherit (as you say, let each endpoint
decide whether it needs it).
- It precludes anynchronously handling that message for much of the same
reasons. e.g., let's say some incoming message requires a long computation
before sending the reply, and you use a thread pool for that. With
{{sender.send()}}, you'd need to build correlation logic into your application.
If you have something like {{message.reply()}} instead, you don't need to - the
RPC layer handles correlations and timing out if a reply doesn't arrive.
Granted, I have no idea how you'd implement that second bullet in akka, and
perhaps for that reason I don't think Spark currently uses that pattern
anywhere. But it sounds like a useful feature to have.
> Standardize internal RPC interface
> ----------------------------------
>
> Key: SPARK-5124
> URL: https://issues.apache.org/jira/browse/SPARK-5124
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Reporter: Reynold Xin
> Assignee: Shixiong Zhu
> Attachments: Pluggable RPC - draft 1.pdf, Pluggable RPC - draft 2.pdf
>
>
> In Spark we use Akka as the RPC layer. It would be great if we can
> standardize the internal RPC interface to facilitate testing. This will also
> provide the foundation to try other RPC implementations in the future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
