I really like the model that Voldemort uses for their protocol buffers
implementation. I ported their client to .net and it was really
simple. The framing is really simple as it prefixes an integer value
before binary data. The binary data is a request made with a protocol
buffer, and so is the response. The response is a protocol buffer
package specific to the method called. They used enums for the method
names. For streaming results it prefixes a 1 if there is another
record and either a 0 or -1 (I forget and need to look at the code.
You get the idea) if it's the end of the stream. I like the idea of
keeping things as simple as possible because it makes it easier for
additional languages to be added quickly. Personally I would prefer to
have an easy request then response model that blocks on the client
side. For simplicity a client could just block and wait for the data
to return or the client could be more advanced and use callbacks or io
completion ports. This would allow a client to use things like
connection pooling to handle concurrency of multiple requests.
As for routing I typically don't like using a proxy because that
limits you to the interface bandwidth of the proxy appliance. For
example if you use something like a netscaler as your proxy in front
of your back end servers. You will have 50 back end servers with gig
connections trying to feed a couple netscalers that can constrain the
bandwidth. Adding more bandwidth means adding more proxy boxes which
gets expensive fast especially with netscalers. Voldemort uses the
concept of node banning. If a node doesn't respond quickly enough or
has errors it will get banned for a period of time. Couple this with
something like SRV records in DNS and you can easily manipulate the
direction of your traffic without using a proxy saving you some cash.
I'm currently working on the .net port of avro and the rpc
implementation is one of my priorities. My goal is to get something to
the point where I can have a utility that connects to the rpc server,
gets protocol handshake, then builds out strongly typed code that a
developer can work against.
{
"namespace": "com.acme",
"protocol": "HelloWorld",
"doc": "Protocol Greetings",
"types": [
{"name": "Greeting", "type": "record", "fields": [
{"name": "message", "type": "string"}]},
{"name": "Curse", "type": "error", "fields": [
{"name": "message", "type": "string"}]}
],
"messages": {
"hello": {
"doc": "Say hello.",
"request": [{"name": "greeting", "type": "Greeting" }],
"response": "Greeting",
"errors": ["Curse"]
}
}
}
would generate
namespace com.acme
{
public class HelloWorld:Avro.Protocol
{
public Greeting hello(Greeting greeting)
}
}
On Thu, Apr 8, 2010 at 3:43 PM, Doug Cutting <[email protected]> wrote:
> Bruce,
>
> Overall this looks like a good approach to me.
>
> How do you anticipate allocating channels? I'm guessing this would be one
> per client object, that a pool of open connections to servers would be
> maintained, and creating a new client object would allocate a new channel.
>
> Currently we perform a handshake per request. This is fairly cheap and
> permits things like routing through proxy servers. Different requests over
> the same connection can talk to different backend servers running different
> versions of the protocol. Also consider the case where, between calls on an
> object, the connection times out, and a new session is established and a new
> handshake must take place.
>
> That said, having a session where the handshake can be assumed vastly
> simplifies one-way messages. Without a response or error on which to prefix
> a handshake response, a one-way client has no means to know that the server
> was able to even parse its request. Yet we'd still like a handshake for
> one-way messages, so that clients and servers need not be versioned in
> lockstep. So the handshake-per-request model doesn't serve one-way messages
> well.
>
> How can we address both of these needs: to permit flexible payload routing
> and efficient one-way messaging?
>
> Doug
>
> Bruce Mitchener wrote:
>>
>> * Think about adding something for true one-way messages, but an empty
>> reply frame is probably sufficient, since that still allows reporting
>> errors
>> if needed (or desired).
>
