@Benh Thanks for the informative/well-documented email regarding wire protocol. Just got through reading and understanding your explanation. I think the proposed patches are a good starting point. I will give them a try.
@Kevints Great example on the JVM. It provides perfect way to start with this. Thanks On Tue, Apr 15, 2014 at 11:06 AM, Vladimir Vivien <vladimir.viv...@gmail.com > wrote: > @benh thanks for the info. > > > On Mon, Apr 14, 2014 at 12:50 PM, Benjamin Hindman <b...@eecs.berkeley.edu > > wrote: > >> Not the Event/Call stuff but the 'libprocess-from' and '202 Accepted' >> reviews. >> >> >> On Mon, Apr 14, 2014 at 10:40 AM, Kevin Sweeney >> <kevin.t.swee...@gmail.com>wrote: >> >> > Didn't see the Event/Call review so current version uses the >> undocumented >> > internal API. Should be straightforward to migrate though >> > >> > >> > On Monday, April 14, 2014, Benjamin Hindman <b...@eecs.berkeley.edu> >> > wrote: >> > >> >> Very cool Kevin. Was this using the patches on Review Board I linked? >> >> >> >> >> >> On Sun, Apr 13, 2014 at 11:28 PM, Kevin Sweeney < >> >> kevin.t.swee...@gmail.com> wrote: >> >> >> >> I've got the start of a JVM verson on github - it can currently >> register >> >> a framework and parse the response. Client-side I still need to figure >> out >> >> how to properly configure Keep-Alive. Servlet can dispatch messages to >> >> message handlers in a type-safe way and returns a 202 for messages it's >> >> going to handle. Code's a mess currently. >> >> >> >> https://github.com/kevints/mesos-framework-api >> >> >> >> >> >> On Fri, Apr 11, 2014 at 7:12 PM, Benjamin Hindman < >> b...@eecs.berkeley.edu >> >> > wrote: >> >> >> >> First, my apologies for getting to this party so late. It's great to >> see >> >> people interested in helping create native-language Mesos libraries. >> >> >> >> Vladimir: my presentation was definitely referring to the the low-level >> >> protocol between master, framework (scheduler), slave, and executors. >> I'll >> >> do my best here to clarify how the current protocol works and what we >> need >> >> to do to get it to the point where we can write native-language >> libraries. >> >> (Eventually it would be great to move some of this into documentation >> as >> >> necessary.) >> >> >> >> As Nikita pointed out, the protocol is currently "HTTP-like". As my >> >> presentation describes, think actors and one-way message passing when >> >> considering how the protocol works. >> >> >> >> To send a message an actor POSTs an HTTP request where the actor that >> is >> >> supposed to receive the message is the first component of the request >> path >> >> and the name of the message is the remaining part of the path. To >> >> distinguish one of these "messages" from a normal HTTP request we look >> to >> >> see if the 'User-Agent' is 'libprocess/...'. For example: >> >> >> >> POST /master/mesos.internal.RegisterFrameworkMessage HTTP/1.1 >> >> User-Agent: libprocess/scheduler(1)@10.0.1.7:53523 >> >> >> >> ... represents a message with the name >> >> 'mesos.internal.RegisterFrameworkMessage' destined for the actor >> 'master' >> >> coming from the actor 'scheduler(1)' at 10.0.1.7:53523. If the >> 'master' >> >> actor were to send a message back it would look something like this: >> >> >> >> POST /scheduler(1)/mesos.internal.FrameworkRegisteredMessage HTTP/1.1 >> >> User-Agent: libprocess/master@10.0.1.7:5050 >> >> >> >> So, one-way message passing via HTTP POST. >> >> >> >> The message data is captured as the body of the HTTP request (which can >> >> be specified using _either_ Content-Length or a Transfer-Encoding, and >> as >> >> Nikita points out we use chunked transfer encoding internally). The >> data is >> >> arbitrary and the actor ultimately decides how it wants to "parse" it. >> In >> >> Mesos, 99% of our messages use serialized protobufs, but we also send >> a few >> >> messages with just arbitrary data. All this really means is that >> knowing >> >> the actor and message name is not enough, you also need to know what >> the >> >> body type is supposed to be for that message. In the future we'll >> probably >> >> enable messages with either JSON or serialized protobuf[1] ... for now, >> >> just serialized protobuf. >> >> >> >> Okay, so where does this break down when trying to do this >> >> language-natively? I've had some of this in the works and this >> conversation >> >> has motivated me to publish some reviews addressing the issues: >> >> >> >> (1) We'll need to return a response if one plans to use a native HTTP >> >> library since it'll expect request/response. >> >> https://reviews.apache.org/r/20276 introduces responding with a '202 >> >> Accepted' for these messages (from the HTTP specification, a '202 >> >> Accepted': "The request has been accepted for processing, but the >> >> processing has not been completed. The request might or might not >> >> eventually be acted upon, as it might be disallowed when processing >> >> actually takes place. There is no facility for re-sending a status code >> >> from an asynchronous operation such as this."). >> >> >> >> (2) Most HTTP libraries will set their 'User-Agent' themselves >> >> >> >> >> > >> > -- >> > Sent from Gmail Mobile >> > >> > > > > -- > Vladimir Vivien > -- Vladimir Vivien
