This is a tough and interesting issue.
Let's put aside the whole RPC issue for a moment and look at how code
progressed from C-land to Java-land.
In C, the developer had to check return values from function calls to see
if the function succeeded. That led to ignoring return values or testing
with a nested block of code:
if (succeeded(open_file(name, &file_struct))) {
if (succeeded(do_something_else(...))) {
do work;
} else {
close resources; release memory; return error;
}
} else { close resources; release memory; return error;}
Java improved on this with exceptions and garbage collection:
try {
File f = open_file(name);
try {
Something s = do_something_else(...)
} finally {
close f;
} catch (...) {
// handle inner and outer errors
}
Add ARM (automatic resource management) and you have something that looks a
lot like a supervisor in Erlang (in my opinion)... basically you run all
your code and the exception handlers handle the exceptional situations.
So, this works for both local calls and remote calls (putting aside the
marshalling of parameters and state across address spaces) as long as the
number of threads of execution can be handled by the runtime/operating
system.
But in the JVM, we can no longer build systems that cause threads to block
on the execution of off-process code because we will need a lot more
threads than the OS (because the JVM is native threaded and can have about
4K threads) can provide.
This is where core.async is particularly nice (it's double extra nice in
JavaScript-land where there's only 1 thread, so blocking is verboten).
Basically, rather than having nested call-back functions (like our C-style
nested if statements), core.async re-writes the code such that where there
are blocking calls to a channel, the code is re-written so that it is in
effect a series of call-backs, but the code that the developer writes is
linear.
Put another way, core.async gives us the syntax of a normal flow, but with
the performance of releasing the thread until a result is available and
then continuing the logical computation.
I think the idea of extending the ideas in core.async to accessing remote
systems makes a lot of sense. Call it RPC. Call it something else... but
it's the same concept... you give the developer linear looking code that
"does the right thing" with off-process calls. The right thing being
releasing the current JVM thread until the answer comes back, dealing
correctly with timeouts, and correctly handing failures by releasing
resources as well as invoking appropriate exception handling code
(supervisor?).
If we don't have a nice layer like this, we're stuck writing code like:
(let [v (alt! (some_call_to_external_system_that_returns a channel)
(timeout 1000))]
(if (not (timeout? v)) (do something with v) (raise error)))
Where we are repeating the boilerplate of timeouts and we probably have to
test v against nil (the channel is closed) and exceptions.
Instead, if we have:
(rpc #(handle_error %) 1000 ;; default timeout
(let [v call_to_external_system] (do something with v)))
And we do the same deep walking with the rpc macro that's done with the go
macro so we identify all the remote calls and turn them into the go-based
code with alt! and the timeout boilerplate.
And then we've got a nice rpc system layered on top of core.async that has
default timeouts... and if rpc supports nesting, then we can tune the
timeout for a given call.
So, I think the concepts and the code in core.async lend themselves
directly to building an rpc-style system where the boilerplate for dealing
with invoking off-process resources and getting the results from these
off-process resources with correct timeout, exception handling, and
resource release.
On Fri, Aug 2, 2013 at 6:46 AM, Timothy Baldridge <tbaldri...@gmail.com>wrote:
> RPC ties a local process (and all the memory its currently using) to a
> remote process. It glosses over the fact that that the link between these
> two processes is quite unreliable. In his thesis, Joe Armstrong also points
> that this system is not only susceptible to hardware/network failure, it's
> also very susceptible to programming failures. A bug in your code could
> cause every 100th RPC call to fail, for example.
>
> So instead of all of this, Erlang (or actually Erlang's OTP libraries)
> proposes a different view:
>
> 1) all messages are sent async and unreliable. This is the way networks
> are designed anyways, if you sent a message to a remote system and it gets
> thrown into a queue, you really don't know what happens to that message in
> the end, except by asking the remote system again, and again until you
> finally get an ACK.
>
> 2) If we follow the above model, then we can start programming in a
> fail-fast mode. This is what OTP is based on. Supervisor processes that
> reset dead sub processes. This is also why I'm not a fan of populating
> error messages across channels. Instead, errors should kill go blocks, and
> then those blocks should be restarted by supervisors.
>
> So RPC assumes that every call will succeed. Message passing systems
> assume that it's kind-of-sort-of-not-really-likely that your message will
> arrive at the remote system. It's a pessimistic view of the world. And with
> the systems I work with, this is the best approach.
>
> So I guess this is a super long way of saying I love the OTP style and
> would love to see it ported to core.async. But RPC is not the way, blocking
> send/recv is not the way. To get reliable systems you need your system to
> always be capable of forward progress, and having a local process tightly
> coupled to a remote process will not get you there.
>
> Timothy
>
>
> On Thu, Aug 1, 2013 at 1:55 PM, ToBeReplaced <toberepla...@gmail.com>wrote:
>
>> A client could use a timeout channel as its receive channel to get
>> timeouts on a blocking call, though I don't think that's the right way to
>> do it. Alternatively, implementing a blocking call with an optional
>> timeout wouldn't be difficult, it just can't be the default.
>>
>> I think if you disallowed nil as a response, then it would be easy to use
>> a variety of different blocking calls -- wait forever, wait 30 seconds,
>> wait until (f) returns truthy, etc.
>>
>> -ToBeReplaced
>>
>>
>> On 08/01/2013 03:36 PM, Cedric Greevey wrote:
>>
>> On Thu, Aug 1, 2013 at 1:09 PM, <toberepla...@gmail.com> wrote:
>>
>>> There would be no intent to solve the messenger problem explicitly --
>>> those semantics are up to the user. By default, in the case of server
>>> death, the client side will just no longer receive responses on its
>>> receive-channel. In the case of a blocking call, this means that the
>>> client side will hang.
>>>
>>
>> Ugh. At the *very* least it should eventually return with some kind of
>> a timeout exception.
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Clojure" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/clojure/P95cOfuXBUs/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> clojure+unsubscr...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
Telegram, Simply Beautiful CMS https://telegr.am
Lift, the simply functional web framework http://liftweb.net
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
