And I've started a skeleton implementation of the Mesos framework API at https://github.com/wickman/pesos
While I vendored the translated protobuf, it would be great to tackle https://issues.apache.org/jira/browse/MESOS-857 at some point. On Sun, Apr 27, 2014 at 12:50 PM, Brian Wickman <wick...@gmail.com> wrote: > I've implemented a pure python version of the basic > Process/ProtobufProcess mechanics and wire protocol. I haven't gone so far > as Kevin and tried to register a framework or maybe talk to a replicated > log, but it shouldn't be much more work: > > https://github.com/wickman/compactor > > Zero documentation but you can read the tests for a general idea of what's > going on. > > cheers, > brian > > > > On Sun, Apr 13, 2014 at 10:28 PM, Kevin Sweeney <kevin.t.swee...@gmail.com > > wrote: > >> I've got the start of a JVM verson on github - it can currently register >> a framework and parse the response. Client-side I still need to figure out >> how to properly configure Keep-Alive. Servlet can dispatch messages to >> message handlers in a type-safe way and returns a 202 for messages it's >> going to handle. Code's a mess currently. >> >> https://github.com/kevints/mesos-framework-api >> >> >> On Fri, Apr 11, 2014 at 7:12 PM, Benjamin Hindman <b...@eecs.berkeley.edu >> > wrote: >> >>> First, my apologies for getting to this party so late. It's great to see >>> people interested in helping create native-language Mesos libraries. >>> >>> Vladimir: my presentation was definitely referring to the the low-level >>> protocol between master, framework (scheduler), slave, and executors. I'll >>> do my best here to clarify how the current protocol works and what we need >>> to do to get it to the point where we can write native-language libraries. >>> (Eventually it would be great to move some of this into documentation as >>> necessary.) >>> >>> As Nikita pointed out, the protocol is currently "HTTP-like". As my >>> presentation describes, think actors and one-way message passing when >>> considering how the protocol works. >>> >>> To send a message an actor POSTs an HTTP request where the actor that is >>> supposed to receive the message is the first component of the request path >>> and the name of the message is the remaining part of the path. To >>> distinguish one of these "messages" from a normal HTTP request we look to >>> see if the 'User-Agent' is 'libprocess/...'. For example: >>> >>> POST /master/mesos.internal.RegisterFrameworkMessage HTTP/1.1 >>> User-Agent: libprocess/scheduler(1)@10.0.1.7:53523 >>> >>> ... represents a message with the name >>> 'mesos.internal.RegisterFrameworkMessage' destined for the actor 'master' >>> coming from the actor 'scheduler(1)' at 10.0.1.7:53523. If the 'master' >>> actor were to send a message back it would look something like this: >>> >>> POST /scheduler(1)/mesos.internal.FrameworkRegisteredMessage HTTP/1.1 >>> User-Agent: libprocess/master@10.0.1.7:5050 >>> >>> So, one-way message passing via HTTP POST. >>> >>> The message data is captured as the body of the HTTP request (which can >>> be specified using _either_ Content-Length or a Transfer-Encoding, and as >>> Nikita points out we use chunked transfer encoding internally). The data is >>> arbitrary and the actor ultimately decides how it wants to "parse" it. In >>> Mesos, 99% of our messages use serialized protobufs, but we also send a few >>> messages with just arbitrary data. All this really means is that knowing >>> the actor and message name is not enough, you also need to know what the >>> body type is supposed to be for that message. In the future we'll probably >>> enable messages with either JSON or serialized protobuf[1] ... for now, >>> just serialized protobuf. >>> >>> Okay, so where does this break down when trying to do this >>> language-natively? I've had some of this in the works and this conversation >>> has motivated me to publish some reviews addressing the issues: >>> >>> (1) We'll need to return a response if one plans to use a native HTTP >>> library since it'll expect request/response. >>> https://reviews.apache.org/r/20276 introduces responding with a '202 >>> Accepted' for these messages (from the HTTP specification, a '202 >>> Accepted': "The request has been accepted for processing, but the >>> processing has not been completed. The request might or might not >>> eventually be acted upon, as it might be disallowed when processing >>> actually takes place. There is no facility for re-sending a status code >>> from an asynchronous operation such as this."). >>> >>> (2) Most HTTP libraries will set their 'User-Agent' themselves, so >>> https://reviews.apache.org/r/20277 introduces a 'libprocess-from' >>> header that works similar to User-Agent. There is still some cleanup I'd >>> love to do around stringification of PIDs (the underlying type Mesos uses >>> for remote actors, inspired by Erlang). Until then, the 'libprocess-from' >>> string is unfortunately esoteric (see the test). >>> >>> The combination of these two patches should make sending and receiving >>> messages straightforward. However, we still plan to expose the low-level >>> Event and Call protobuf messages and that will be the preferred approach >>> for building a native-language library. Follow along at >>> https://issues.apache.org/jira/browse/MESOS-1127 for more details. (To >>> be clear, you'd still be able to implement native-language libraries with >>> the patches above but we'll be deprecating the protobufs you'd be using in >>> favor of Event and Call protobufs instead. If you're eager to get that >>> going before Event and Call are committed I'm happy to discuss the existing >>> protobufs in more detail.) >>> >>> I hope this helps. >>> >>> Ben. >>> >>> >>> On Fri, Apr 11, 2014 at 4:54 AM, Vladimir Vivien < >>> vladimir.viv...@gmail.com> wrote: >>> >>>> Nikita >>>> Thanks for the JIRA. >>>> >>>> >>>> On Wed, Apr 9, 2014 at 2:16 PM, Vetoshkin Nikita < >>>> nikita.vetosh...@gmail.com >>>> > wrote: >>>> >>>> > BTW, there is also somehow related ticket >>>> > https://issues.apache.org/jira/browse/MESOS-930 >>>> > >>>> > >>>> > On Wed, Apr 9, 2014 at 9:54 PM, Benjamin Mahler >>>> > <benjamin.mah...@gmail.com>wrote: >>>> > >>>> > > > >>>> > > > I thought the low-level api being referred in the >>>> > > > video had to do with communication between master and >>>> > framework|executor >>>> > > > for scheduling. But, it's really administrative. I thought that >>>> would >>>> > > > have been an opportunity for a Go binding that did not require >>>> the C++ >>>> > > > libraries. >>>> > > > >>>> > > >>>> > > Vladimir, the low-level API referred to in the talk is exactly what >>>> > you're >>>> > > interpreting, it is for communication between master and scheduler, >>>> and >>>> > > slave and executor. You could definitely build pure go bindings as >>>> you >>>> > > described, just not with JSON. >>>> > > >>>> > > Forget I mentioned anything about the administrative endpoints and >>>> JSON, >>>> > as >>>> > > I see that's leading to confusion. ;) >>>> > > >>>> > > On Wed, Apr 9, 2014 at 3:39 AM, Vladimir Vivien >>>> > > <vladimir.viv...@gmail.com>wrote: >>>> > > >>>> > > > Ben, >>>> > > > Thank you for clarifying. I thought the low-level api being >>>> referred in >>>> > > the >>>> > > > video had to do with communication between master and >>>> > framework|executor >>>> > > > for scheduling. But, it's really administrative. I thought that >>>> would >>>> > > > have been an opportunity for a Go binding that did not require >>>> the C++ >>>> > > > libraries. >>>> > > > >>>> > > > Thanks anyway. >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > On Tue, Apr 8, 2014 at 4:52 PM, Benjamin Mahler >>>> > > > <benjamin.mah...@gmail.com>wrote: >>>> > > > >>>> > > > > Sorry, I was not referring to implementing a scheduler via JSON >>>> > instead >>>> > > > of >>>> > > > > protobuf, in theory that would be possible but there has been no >>>> > > planning >>>> > > > > in this area. Sorry for the confusion. >>>> > > > > >>>> > > > > I was referring to administrative endpoints. For example, >>>> kicking a >>>> > > > > framework out or telling the master a slave is needs to be >>>> repaired. >>>> > > > These >>>> > > > > endpoints may rely on the ability to convert JSON to internal >>>> > > protobufs. >>>> > > > > >>>> > > > > Can you clarify what you're looking to do? Are you looking to >>>> > implement >>>> > > > an >>>> > > > > API in Go that communicates with JSON instead of serialized >>>> protobuf? >>>> > > > > >>>> > > > > On Tue, Apr 8, 2014 at 1:19 PM, Vladimir Vivien >>>> > > > > <vladimir.viv...@gmail.com>wrote: >>>> > > > > >>>> > > > > > Ben, >>>> > > > > > That is exactly what I am asking. >>>> > > > > > Is that something coming up soon, is there a JIRA I can look >>>> at? >>>> > > > > > I wanna get early start on a native json Go api or even help >>>> out if >>>> > > > > > possible. >>>> > > > > > >>>> > > > > > >>>> > > > > > On Tue, Apr 8, 2014 at 3:25 PM, Benjamin Mahler >>>> > > > > > <benjamin.mah...@gmail.com>wrote: >>>> > > > > > >>>> > > > > > > +vinod, benh >>>> > > > > > > >>>> > > > > > > Hey Vladimir, there will be some authenticated REST >>>> endpoints at >>>> > > some >>>> > > > > > > point, there is some work in this area underway. >>>> > > > > > > >>>> > > > > > > We have the ability to encode protobuf messages as JSON, so >>>> the >>>> > > plan >>>> > > > > was >>>> > > > > > to >>>> > > > > > > have any REST endpoints directly use JSON to send us >>>> protobuf >>>> > > > messages. >>>> > > > > > I'm >>>> > > > > > > not sure if this is what you're asking though? >>>> > > > > > > >>>> > > > > > > >>>> > > > > > > On Tue, Apr 8, 2014 at 11:13 AM, Vetoshkin Nikita < >>>> > > > > > > nikita.vetosh...@gmail.com> wrote: >>>> > > > > > > >>>> > > > > > > > I'm not a mesos guy, just very curious. But in my opinion >>>> - I >>>> > > doubt >>>> > > > > it, >>>> > > > > > > > HTTP is synchronous request-response protocol. Mesos needs >>>> > > > something >>>> > > > > > more >>>> > > > > > > > robust for message passing. Websockets anyone? :) >>>> > > > > > > > >>>> > > > > > > > >>>> > > > > > > > On Tue, Apr 8, 2014 at 10:08 PM, Vladimir Vivien >>>> > > > > > > > <vladimir.viv...@gmail.com>wrote: >>>> > > > > > > > >>>> > > > > > > > > Ben / Nikita >>>> > > > > > > > > Thanks for the pointers. >>>> > > > > > > > > So, (without digging yet) is it a fair summary to say >>>> that >>>> > > > > libprocess >>>> > > > > > > > wraps >>>> > > > > > > > > protobufs-encoded calls and push them over HTTP to >>>> > > master/slaves >>>> > > > ? >>>> > > > > > Will >>>> > > > > > > > > protobuf (eventually) be supplanted by direct HTTP via >>>> REST >>>> > or >>>> > > > > > similar >>>> > > > > > > ? >>>> > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > On Mon, Apr 7, 2014 at 2:54 PM, Vetoshkin Nikita < >>>> > > > > > > > > nikita.vetosh...@gmail.com >>>> > > > > > > > > > wrote: >>>> > > > > > > > > >>>> > > > > > > > > > Or, just to get to know - you can take tcpdump and >>>> take a >>>> > > look >>>> > > > :) >>>> > > > > > > > > > >>>> > > > > > > > > > I personally wouldn't call that HTTP. Something >>>> "HTTP-like" >>>> > > > would >>>> > > > > > > > > describe >>>> > > > > > > > > > it better. Because it's not request-response. It's >>>> just >>>> > > message >>>> > > > > > > > passing, >>>> > > > > > > > > no >>>> > > > > > > > > > need to wait for the answer - send new message one >>>> after >>>> > > > another. >>>> > > > > > > Every >>>> > > > > > > > > > message is POST with address and message type encoded >>>> in >>>> > URI: >>>> > > > > POST >>>> > > > > > > > > > /executor(1)/mesos.internal.RunTaskMessage. Sender is >>>> > encoded >>>> > > > in >>>> > > > > > > > > User-Agent >>>> > > > > > > > > > header, e.g: libprocess/slave(1)@127.0.0.1:5051. Body >>>> > > contains >>>> > > > > > > > protobuf >>>> > > > > > > > > > message, Transfer-Encoding is always "chunked". >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > > > > On Mon, Apr 7, 2014 at 10:42 PM, Benjamin Mahler >>>> > > > > > > > > > <benjamin.mah...@gmail.com>wrote: >>>> > > > > > > > > > >>>> > > > > > > > > > > Unfortunately you will need to learn this by >>>> looking at >>>> > the >>>> > > > > code >>>> > > > > > in >>>> > > > > > > > > > > libprocess, as the message passing format is not >>>> > explicitly >>>> > > > > > > > documented >>>> > > > > > > > > at >>>> > > > > > > > > > > the current time. >>>> > > > > > > > > > > >>>> > > > > > > > > > > Start with calls like ProtobufProcess::send() and >>>> dig >>>> > your >>>> > > > way >>>> > > > > > > down. >>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > > On Sat, Apr 5, 2014 at 7:52 AM, Vladimir Vivien >>>> > > > > > > > > > > <vladimir.viv...@gmail.com>wrote: >>>> > > > > > > > > > > >>>> > > > > > > > > > > > I was watching this video from >>>> > > > > > > > > > > > https://www.youtube.com/watch?v=n5GT7OFSh58fromBen >>>> > > where >>>> > > > he >>>> > > > > > > > talked >>>> > > > > > > > > > > > about the wire protocol for Mesos being done in >>>> > > > > > > > > > > > HTTP. >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > Where can I learn about the low-level wire >>>> protocol >>>> > > either >>>> > > > in >>>> > > > > > > > > > > documentation >>>> > > > > > > > > > > > or browsing through the code. >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > Thanks. >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > -- >>>> > > > > > > > > > > > Vladimir Vivien >>>> > > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > -- >>>> > > > > > > > > Vladimir Vivien >>>> > > > > > > > > >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > -- >>>> > > > > > Vladimir Vivien >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > -- >>>> > > > Vladimir Vivien >>>> > > > >>>> > > >>>> > >>>> >>>> >>>> >>>> -- >>>> Vladimir Vivien >>>> >>> >>> >> >
