I'd like to warn those of you using the cvs-gateway that the lsh
source code is in some flux right now, and it will probably not work
for the next few days.
And I'd like to explain what I'm doing now. Keep in mind that this is
work in progress; the draft below describes how I think about it now;
details may well change.
A problem in lsh is that the logical control flow, say "first connect,
then listen on some local socket. When someone connects, request a tcp
channel to the other end, and when the channel is opened successfully,
start forwarding data" is more or less orthogonal to the control flow
of the C program. This is caused by the "asyncronous" or
"event-driven" model, where all action happen as a response to some
i/o event. When sending a channel-open request, for example, it is not
possible to simply wait for the answer. One *must* return to the
main select loop and somehow arrange for an appropriate callback to be
invoked when the answer is ready.
Therefore, when programming non-trivial actions in lsh, one is forced
to use a programming style with explicit continuations: You stuff all
the relevant state inte some callback structure, register it to be
called at some later time, and then return control to the main select
loop.
This is doable, but highly inconvenient. So we should not have to do
it, at least not by hand. To make this easier, it would be nice to
write the control parts in a different language, and generate all
hooks needed for the ayncronous machinery automatically. I.e. we'd
like to program the thing as if we could call a function, and have the
function call return later, and have all needed callbacks and
continuation structures done someplace out of sight.
To do this, primitive commands on the lsh machinery are still
written in C. A command takes one argument, and returns one value
(both represented as struct lsh_object *). A command is an object with
a method
struct lsh_object *call(struct command *self,
struct lsh_object *argument,
struct command_continuation *c)
It should start doing the operation (and return to the main loop), and
make sure that when finished, the continuation c is invoked with the
result of the operation. This is very similar to many of the existing
lsh abstractions and callbacks.
But now we use a second language for combining these primitive
commands to more complex commands. An example, relevant to tcp
forwarding:
(lambda (port connection)
(start-io (listen port connection)
(open-direct-tcp connection)))
Everything is executed from left to right. Functions really have only
one argument; but a function that takes one argument and returns a new
function accepting the next argument acts a lot like a function of two
arguments. To be precise, (a b c) is a shorthand for ((a b) c) and
(lambda (a b) c) is a shorthand for (lambda (a) (lambda (b) c)), but
that is not very important now.
The primitive operations above are
(listen port connection): Binds a port, and associates it with the
connection (so that it is closed automatically if the connection
dies). It returns a file object every time someone connects to the
port. Yes, that is correct: A command can return several times, and
this function should do that. It will be the listen_callback on the
bound socket that will invoke the continuation of this command.
(open-direct-tcp connection): Send a channel-open message requesting a
channel of type direct-tcp. Returns a channel object, or NULL if the
server at the other end of the connection refused to open the channel.
(start-io file channel): Connects a file object to a channel.
These three primitives should be easy to implement using the existing
lsh abstractions. (We need some queues in the channel and
channel_table objects to keep track of pending requests, but that is
quite simple).
Taking into acount the evaluation rules, what the expression above
means is:
1. First we must wait until we have an ssh-connection.
2. Next, we bind the port, and wait for somebody to connect.
3. When we get a connection, we request a tcp-direct channel, and
wait for the response.
4. When the channel is opened successfully, we start copying data
between the socket and the channel.
I think this way to describe what should be done and when is concise
and fairly readable. Definitely a lot better than hand coding dozens
of continuation structures in C.
I have extended the make_class program to process such expressions (I
also changed the magic word from "CLASS" to "GABA" to make it more
unique, and because this feature doesn't have much to do with OO):
/* GABA:
(expr
(name local_tcp_forward)
(globals
(start-io foo_start)
(listen foo_listen)
(open-direct-tcpip foo_open))
(expr (lambda (port connection)
(start-io (listen port connection)
(open-direct-tcpip connection)))))
*/
(The foo_xxx symbols should be replaced with the names of the
corresponding primitive commands). The make class script generates a
function local_tcp_forward() which evaluates the given expression and
returns its value, which is the more complex command. (This evaluation is
(currently) done without any continuation stuff, so it will break if it
needs to call any function that would block).
The current implementation transforms the expression into a very
unreadable "combinator expression" using the S, K and I combinators;
these combinators are implemented in C as primitive commands. The
above translates to
A(S2(A(S2(A(K1, S)), A(S2(A(K1, S2(A(K1, foo_start)))), A(S2(A(K1, S2(foo_listen))),
K)))), A(S2(A(K1, K)), foo_open));
This representation is very easy to compile to, and easy to execute.
One nice feature is that all variables (in the example above, the
variables are the formal parameters PORT and CONNECTION) are
completely eliminated. It's main drawback is the size of the generated
expressions, but I think it should be good enough for simple commands
like this.
If it turns out to be too unwieldy or inefficient, it can be replaced
without effecting the main lsh source code much. Reasonable
alternatives include: A more optimized combinator translator, an
explicit interpreter (with variables), and compilation directly to
continuation passing style C code.
As you probably guessed by now, I think this approach is pretty cool
;) We'll see how it turns out.
Happy hacking,
/Niels
