Ben M, Doh! You are right, I am sending the wrong message (went one level too deep).
Thanks for the help. On Fri, May 9, 2014 at 2:38 PM, Benjamin Mahler <benjamin.mah...@gmail.com>wrote: > It seems as though you're trying to send a 'RegisterFrameworkMessage' but > you're actually encoding a 'FrameworkInfo'? > > The error message indicates that your binary data is not matching the > expected format. > > > On Fri, May 9, 2014 at 4:29 AM, Vladimir Vivien <vladimir.viv...@gmail.com > > wrote: > >> Ben M, >> Re built mesos from master. Made the suggested changes for headers. >> Getting HTTP response back now. >> >> Still getting Error, even though getting 201 back (see below) >> "... protobuf.hpp:400] Initialization errors: framework.user, >> framework.name". >> >> I am sending protobuf encoded binary data. >> I am using protobuf 2.5 + a Go protobuf lib. >> >> The String representation is >> "user:"test" name:"gomes" id:<value:"gomes-framework-1" >" >> >> HTTP Request >> POST /master/mesos.internal.RegisterFrameworkMessage HTTP/1.1 >> Host: 127.0.0.1:5050 >> User-Agent: Go 1.1 package http >> Content-Length: 34 >> Connection: Keep-Alive >> Content-Type: application/x-protobuf >> Libprocess-From: scheduler(1)@127.0.0.1:8080 >> Accept-Encoding: gzip >> >> HTTP Response >> 2014/05/09 07:27:11 Response Body: >> HTTP/1.1 202 Accepted >> Content-Length: 0 >> Date: Fri, 09 May 2014 11:27:11 GMT >> >> >> >> On Tue, May 6, 2014 at 4:06 PM, Benjamin Mahler < >> benjamin.mah...@gmail.com> wrote: >> >>> Can you build this off of the latest master code? The reviews have been >>> committed! >>> >>> You should now use "Libprocess-From" instead of "User-Agent". >>> >>> E.g. >>> >>> Libprocess-From: scheduler@127.0.0.1:8080 >>> >>> Also, what is your "string" representation of the protobuf? You're >>> serializing it using a protobuf library, right? It should be binary data. >>> >>> >>> On Sun, May 4, 2014 at 7:57 AM, Vladimir Vivien < >>> vladimir.viv...@gmail.com> wrote: >>> >>>> Slow progress from Go front: >>>> I am at a point where I am testing this approach using Go. >>>> >>>> HTTP Sent: >>>> POST /master/mesos.internal.RegisterFrameworkMessage HTTP/1.1 >>>> Host: 127.0.0.1:5050 >>>> User-Agent: libprocess/scheduler(1)@127.0.0.1:8080 >>>> Content-Length: 34 >>>> Connection: Keep-Alive >>>> Content-Type: application/x-protobuf >>>> Accept-Encoding: gzip >>>> >>>> String representation of protobuf sent: >>>> user:"test" name:"gomes" id:<value:"gomes-framework-1" > >>>> >>>> Master console response >>>> W0504 10:53:48.263000 3960 protobuf.hpp:399] Initialization errors: >>>> framework.user, framework.name >>>> >>>> Before I delve in to CPP to see what's going on, hope someone can let >>>> me know what I am doing wrong. >>>> >>>> PS. I havent applied the two patches mentioned here. >>>> >>>> vladimir.vivien >>>> >>>> >>>> >>>> On Wed, Apr 30, 2014 at 1:59 AM, Benjamin Mahler < >>>> benjamin.mah...@gmail.com> wrote: >>>> >>>>> Brian, I'm curious whether most python HTTP libraries support using >>>>> custom HTTP request headers. >>>>> >>>>> We'll land benh's patches to send 202 responses in 0.19.0: >>>>> >>>>> https://reviews.apache.org/r/20276/ >>>>> https://reviews.apache.org/r/20277/ >>>>> >>>>> Once these get committed, you should be able to get something working >>>>> nicely. This will allow libraries like "pesos" to use a custom header >>>>> "Libprocess-From" to receive 202 responses for messages. >>>>> >>>>> We'd also like to get some more foundation laid for the lower-level >>>>> API, I'll let benh comment on what else is needed here: >>>>> >>>>> https://reviews.apache.org/r/20309/ >>>>> >>>>> >>>>> On Sun, Apr 27, 2014 at 9:23 PM, Vetoshkin Nikita < >>>>> nikita.vetosh...@gmail.com> wrote: >>>>> >>>>>> That's great news indeed! >>>>>> Maybe there should be a place on mesos site mentioning such efforts? >>>>>> >>>>>> >>>>>> On Mon, Apr 28, 2014 at 5:24 AM, Brian Wickman <wick...@gmail.com> >>>>>> wrote: >>>>>> >>>>>> > To my knowledge there is no Zookeeper Group implementation for Go >>>>>> yet. >>>>>> > There do however appear to be Go zookeeper >>>>>> > bindings<https://github.com/samuel/go-zookeeper>. >>>>>> > From my day of spelunking through code, it seems that probably the >>>>>> most >>>>>> > challenging thing will be getting a ZookeeperMasterDetector >>>>>> implementation >>>>>> > working correctly to enable Scheduler "high availability" mode for >>>>>> these >>>>>> > other languages. >>>>>> > >>>>>> > Java/Python/C++ have the benefit of preexisting Group >>>>>> implementations >>>>>> > (java< >>>>>> > >>>>>> https://github.com/twitter/commons/blob/master/src/java/com/twitter/common/zookeeper/Group.java >>>>>> > > >>>>>> > , python< >>>>>> > >>>>>> https://github.com/twitter/commons/blob/master/src/python/twitter/common/zookeeper/group/kazoo_group.py >>>>>> > > >>>>>> > , c++< >>>>>> > >>>>>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob;f=src/zookeeper/group.cpp;h=70972980dd25618c76eb36e0a31d060c096e36dc;hb=HEAD >>>>>> > >) >>>>>> > which are the foundation for ZK master detection. An intermediate >>>>>> option >>>>>> > could be to pull the ZookeeperMasterDetectorProcess out into a >>>>>> standalone >>>>>> > binary and have the native implementation talk to it as a sort of >>>>>> proxy to >>>>>> > figure out who the leading master is. But that's not nearly as >>>>>> satisfying >>>>>> > as a pure language implementation. >>>>>> > >>>>>> > ~brian >>>>>> > >>>>>> > >>>>>> > On Sun, Apr 27, 2014 at 3:22 PM, Vladimir Vivien >>>>>> > <vladimir.viv...@gmail.com>wrote: >>>>>> > >>>>>> > > It must be the right time for this. I started the same effort >>>>>> for Go. >>>>>> > > I am following Kevin's example and Ben's well-documented email >>>>>> (in this >>>>>> > > thread). >>>>>> > > Very early but should have some proof-of-concept working soon. >>>>>> > > >>>>>> > > https://github.com/vladimirvivien/ionos >>>>>> > > >>>>>> > > >>>>>> > > On Sun, Apr 27, 2014 at 6:07 PM, Tom Arnfeld <t...@duedil.com> >>>>>> wrote: >>>>>> > > >>>>>> > >> (Sorry to jump into this thread) >>>>>> > >> >>>>>> > >> I have to say, this is very exciting! It only became apparent to >>>>>> me how >>>>>> > >> painful it is having to compile the mesos egg into a Python >>>>>> framework >>>>>> > the >>>>>> > >> past week >>>>>> > >> while writing one, especially when running the framework on a >>>>>> different >>>>>> > >> architecture to the one running the executor! >>>>>> > >> >>>>>> > >> Looking forward to being able to use this. :-) >>>>>> > >> >>>>>> > >> It’d be awesome if the API was near identical, so one could >>>>>> switch >>>>>> > >> between `mesos` and `pesos` easily… >>>>>> > >> >>>>>> > >> Tom. >>>>>> > >> >>>>>> > >> On 27 Apr 2014, at 22:42, Brian Wickman <wick...@gmail.com> >>>>>> wrote: >>>>>> > >> >>>>>> > >> > And I've started a skeleton implementation of the Mesos >>>>>> framework API >>>>>> > at >>>>>> > >> > https://github.com/wickman/pesos >>>>>> > >> > >>>>>> > >> > While I vendored the translated protobuf, it would be great to >>>>>> tackle >>>>>> > >> > https://issues.apache.org/jira/browse/MESOS-857 at some point. >>>>>> > >> > >>>>>> > >> > >>>>>> > >> > On Sun, Apr 27, 2014 at 12:50 PM, Brian Wickman < >>>>>> wick...@gmail.com> >>>>>> > >> wrote: >>>>>> > >> > >>>>>> > >> >> I've implemented a pure python version of the basic >>>>>> > >> >> Process/ProtobufProcess mechanics and wire protocol. I >>>>>> haven't gone >>>>>> > >> so far >>>>>> > >> >> as Kevin and tried to register a framework or maybe talk to a >>>>>> > >> replicated >>>>>> > >> >> log, but it shouldn't be much more work: >>>>>> > >> >> >>>>>> > >> >> https://github.com/wickman/compactor >>>>>> > >> >> >>>>>> > >> >> Zero documentation but you can read the tests for a general >>>>>> idea of >>>>>> > >> what's >>>>>> > >> >> going on. >>>>>> > >> >> >>>>>> > >> >> cheers, >>>>>> > >> >> brian >>>>>> > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> > >> >> On Sun, Apr 13, 2014 at 10:28 PM, Kevin Sweeney < >>>>>> > >> kevin.t.swee...@gmail.com >>>>>> > >> >>> wrote: >>>>>> > >> >> >>>>>> > >> >>> I've got the start of a JVM verson on github - it can >>>>>> currently >>>>>> > >> register >>>>>> > >> >>> a framework and parse the response. Client-side I still need >>>>>> to >>>>>> > >> figure out >>>>>> > >> >>> how to properly configure Keep-Alive. Servlet can dispatch >>>>>> messages >>>>>> > to >>>>>> > >> >>> message handlers in a type-safe way and returns a 202 for >>>>>> messages >>>>>> > >> it's >>>>>> > >> >>> going to handle. Code's a mess currently. >>>>>> > >> >>> >>>>>> > >> >>> https://github.com/kevints/mesos-framework-api >>>>>> > >> >>> >>>>>> > >> >>> >>>>>> > >> >>> On Fri, Apr 11, 2014 at 7:12 PM, Benjamin Hindman < >>>>>> > >> b...@eecs.berkeley.edu >>>>>> > >> >>>> wrote: >>>>>> > >> >>> >>>>>> > >> >>>> First, my apologies for getting to this party so late. It's >>>>>> great >>>>>> > to >>>>>> > >> see >>>>>> > >> >>>> people interested in helping create native-language Mesos >>>>>> > libraries. >>>>>> > >> >>>> >>>>>> > >> >>>> Vladimir: my presentation was definitely referring to the >>>>>> the >>>>>> > >> low-level >>>>>> > >> >>>> protocol between master, framework (scheduler), slave, and >>>>>> > >> executors. I'll >>>>>> > >> >>>> do my best here to clarify how the current protocol works >>>>>> and what >>>>>> > >> we need >>>>>> > >> >>>> to do to get it to the point where we can write >>>>>> native-language >>>>>> > >> libraries. >>>>>> > >> >>>> (Eventually it would be great to move some of this into >>>>>> > >> documentation as >>>>>> > >> >>>> necessary.) >>>>>> > >> >>>> >>>>>> > >> >>>> As Nikita pointed out, the protocol is currently >>>>>> "HTTP-like". As my >>>>>> > >> >>>> presentation describes, think actors and one-way message >>>>>> passing >>>>>> > when >>>>>> > >> >>>> considering how the protocol works. >>>>>> > >> >>>> >>>>>> > >> >>>> To send a message an actor POSTs an HTTP request where the >>>>>> actor >>>>>> > >> that is >>>>>> > >> >>>> supposed to receive the message is the first component of >>>>>> the >>>>>> > >> request path >>>>>> > >> >>>> and the name of the message is the remaining part of the >>>>>> path. To >>>>>> > >> >>>> distinguish one of these "messages" from a normal HTTP >>>>>> request we >>>>>> > >> look to >>>>>> > >> >>>> see if the 'User-Agent' is 'libprocess/...'. For example: >>>>>> > >> >>>> >>>>>> > >> >>>> POST /master/mesos.internal.RegisterFrameworkMessage >>>>>> HTTP/1.1 >>>>>> > >> >>>> User-Agent: libprocess/scheduler(1)@10.0.1.7:53523 >>>>>> > >> >>>> >>>>>> > >> >>>> ... represents a message with the name >>>>>> > >> >>>> 'mesos.internal.RegisterFrameworkMessage' destined for the >>>>>> actor >>>>>> > >> 'master' >>>>>> > >> >>>> coming from the actor 'scheduler(1)' at 10.0.1.7:53523. If >>>>>> the >>>>>> > >> 'master' >>>>>> > >> >>>> actor were to send a message back it would look something >>>>>> like >>>>>> > this: >>>>>> > >> >>>> >>>>>> > >> >>>> POST /scheduler(1)/mesos.internal.FrameworkRegisteredMessage >>>>>> > HTTP/1.1 >>>>>> > >> >>>> User-Agent: libprocess/master@10.0.1.7:5050 >>>>>> > >> >>>> >>>>>> > >> >>>> So, one-way message passing via HTTP POST. >>>>>> > >> >>>> >>>>>> > >> >>>> The message data is captured as the body of the HTTP >>>>>> request (which >>>>>> > >> can >>>>>> > >> >>>> be specified using _either_ Content-Length or a >>>>>> Transfer-Encoding, >>>>>> > >> and as >>>>>> > >> >>>> Nikita points out we use chunked transfer encoding >>>>>> internally). The >>>>>> > >> data is >>>>>> > >> >>>> arbitrary and the actor ultimately decides how it wants to >>>>>> "parse" >>>>>> > >> it. In >>>>>> > >> >>>> Mesos, 99% of our messages use serialized protobufs, but we >>>>>> also >>>>>> > >> send a few >>>>>> > >> >>>> messages with just arbitrary data. All this really means is >>>>>> that >>>>>> > >> knowing >>>>>> > >> >>>> the actor and message name is not enough, you also need to >>>>>> know >>>>>> > what >>>>>> > >> the >>>>>> > >> >>>> body type is supposed to be for that message. In the future >>>>>> we'll >>>>>> > >> probably >>>>>> > >> >>>> enable messages with either JSON or serialized protobuf[1] >>>>>> ... for >>>>>> > >> now, >>>>>> > >> >>>> just serialized protobuf. >>>>>> > >> >>>> >>>>>> > >> >>>> Okay, so where does this break down when trying to do this >>>>>> > >> >>>> language-natively? I've had some of this in the works and >>>>>> this >>>>>> > >> conversation >>>>>> > >> >>>> has motivated me to publish some reviews addressing the >>>>>> issues: >>>>>> > >> >>>> >>>>>> > >> >>>> (1) We'll need to return a response if one plans to use a >>>>>> native >>>>>> > HTTP >>>>>> > >> >>>> library since it'll expect request/response. >>>>>> > >> >>>> https://reviews.apache.org/r/20276 introduces responding >>>>>> with a >>>>>> > '202 >>>>>> > >> >>>> Accepted' for these messages (from the HTTP specification, >>>>>> a '202 >>>>>> > >> >>>> Accepted': "The request has been accepted for processing, >>>>>> but the >>>>>> > >> >>>> processing has not been completed. The request might or >>>>>> might not >>>>>> > >> >>>> eventually be acted upon, as it might be disallowed when >>>>>> processing >>>>>> > >> >>>> actually takes place. There is no facility for re-sending a >>>>>> status >>>>>> > >> code >>>>>> > >> >>>> from an asynchronous operation such as this."). >>>>>> > >> >>>> >>>>>> > >> >>>> (2) Most HTTP libraries will set their 'User-Agent' >>>>>> themselves, so >>>>>> > >> >>>> https://reviews.apache.org/r/20277 introduces a >>>>>> 'libprocess-from' >>>>>> > >> >>>> header that works similar to User-Agent. There is still some >>>>>> > cleanup >>>>>> > >> I'd >>>>>> > >> >>>> love to do around stringification of PIDs (the underlying >>>>>> type >>>>>> > Mesos >>>>>> > >> uses >>>>>> > >> >>>> for remote actors, inspired by Erlang). Until then, the >>>>>> > >> 'libprocess-from' >>>>>> > >> >>>> string is unfortunately esoteric (see the test). >>>>>> > >> >>>> >>>>>> > >> >>>> The combination of these two patches should make sending and >>>>>> > >> receiving >>>>>> > >> >>>> messages straightforward. However, we still plan to expose >>>>>> the >>>>>> > >> low-level >>>>>> > >> >>>> Event and Call protobuf messages and that will be the >>>>>> preferred >>>>>> > >> approach >>>>>> > >> >>>> for building a native-language library. Follow along at >>>>>> > >> >>>> https://issues.apache.org/jira/browse/MESOS-1127 for more >>>>>> details. >>>>>> > >> (To >>>>>> > >> >>>> be clear, you'd still be able to implement native-language >>>>>> > libraries >>>>>> > >> with >>>>>> > >> >>>> the patches above but we'll be deprecating the protobufs >>>>>> you'd be >>>>>> > >> using in >>>>>> > >> >>>> favor of Event and Call protobufs instead. If you're eager >>>>>> to get >>>>>> > >> that >>>>>> > >> >>>> going before Event and Call are committed I'm happy to >>>>>> discuss the >>>>>> > >> existing >>>>>> > >> >>>> protobufs in more detail.) >>>>>> > >> >>>> >>>>>> > >> >>>> I hope this helps. >>>>>> > >> >>>> >>>>>> > >> >>>> Ben. >>>>>> > >> >>>> >>>>>> > >> >>>> >>>>>> > >> >>>> On Fri, Apr 11, 2014 at 4:54 AM, Vladimir Vivien < >>>>>> > >> >>>> vladimir.viv...@gmail.com> wrote: >>>>>> > >> >>>> >>>>>> > >> >>>>> Nikita >>>>>> > >> >>>>> Thanks for the JIRA. >>>>>> > >> >>>>> >>>>>> > >> >>>>> >>>>>> > >> >>>>> On Wed, Apr 9, 2014 at 2:16 PM, Vetoshkin Nikita < >>>>>> > >> >>>>> nikita.vetosh...@gmail.com >>>>>> > >> >>>>>> wrote: >>>>>> > >> >>>>> >>>>>> > >> >>>>>> BTW, there is also somehow related ticket >>>>>> > >> >>>>>> https://issues.apache.org/jira/browse/MESOS-930 >>>>>> > >> >>>>>> >>>>>> > >> >>>>>> >>>>>> > >> >>>>>> On Wed, Apr 9, 2014 at 9:54 PM, Benjamin Mahler >>>>>> > >> >>>>>> <benjamin.mah...@gmail.com>wrote: >>>>>> > >> >>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> I thought the low-level api being referred in the >>>>>> > >> >>>>>>>> video had to do with communication between master and >>>>>> > >> >>>>>> framework|executor >>>>>> > >> >>>>>>>> for scheduling. But, it's really administrative. I >>>>>> thought >>>>>> > that >>>>>> > >> >>>>> would >>>>>> > >> >>>>>>>> have been an opportunity for a Go binding that did not >>>>>> require >>>>>> > >> >>>>> the C++ >>>>>> > >> >>>>>>>> libraries. >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>> >>>>>> > >> >>>>>>> Vladimir, the low-level API referred to in the talk is >>>>>> exactly >>>>>> > >> what >>>>>> > >> >>>>>> you're >>>>>> > >> >>>>>>> interpreting, it is for communication between master and >>>>>> > >> scheduler, >>>>>> > >> >>>>> and >>>>>> > >> >>>>>>> slave and executor. You could definitely build pure go >>>>>> bindings >>>>>> > as >>>>>> > >> >>>>> you >>>>>> > >> >>>>>>> described, just not with JSON. >>>>>> > >> >>>>>>> >>>>>> > >> >>>>>>> Forget I mentioned anything about the administrative >>>>>> endpoints >>>>>> > and >>>>>> > >> >>>>> JSON, >>>>>> > >> >>>>>> as >>>>>> > >> >>>>>>> I see that's leading to confusion. ;) >>>>>> > >> >>>>>>> >>>>>> > >> >>>>>>> On Wed, Apr 9, 2014 at 3:39 AM, Vladimir Vivien >>>>>> > >> >>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>>> > >> >>>>>>> >>>>>> > >> >>>>>>>> Ben, >>>>>> > >> >>>>>>>> Thank you for clarifying. I thought the low-level api >>>>>> being >>>>>> > >> >>>>> referred in >>>>>> > >> >>>>>>> the >>>>>> > >> >>>>>>>> video had to do with communication between master and >>>>>> > >> >>>>>> framework|executor >>>>>> > >> >>>>>>>> for scheduling. But, it's really administrative. I >>>>>> thought >>>>>> > that >>>>>> > >> >>>>> would >>>>>> > >> >>>>>>>> have been an opportunity for a Go binding that did not >>>>>> require >>>>>> > >> >>>>> the C++ >>>>>> > >> >>>>>>>> libraries. >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> Thanks anyway. >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> On Tue, Apr 8, 2014 at 4:52 PM, Benjamin Mahler >>>>>> > >> >>>>>>>> <benjamin.mah...@gmail.com>wrote: >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>>> Sorry, I was not referring to implementing a scheduler >>>>>> via >>>>>> > JSON >>>>>> > >> >>>>>> instead >>>>>> > >> >>>>>>>> of >>>>>> > >> >>>>>>>>> protobuf, in theory that would be possible but there >>>>>> has been >>>>>> > no >>>>>> > >> >>>>>>> planning >>>>>> > >> >>>>>>>>> in this area. Sorry for the confusion. >>>>>> > >> >>>>>>>>> >>>>>> > >> >>>>>>>>> I was referring to administrative endpoints. For >>>>>> example, >>>>>> > >> >>>>> kicking a >>>>>> > >> >>>>>>>>> framework out or telling the master a slave is needs >>>>>> to be >>>>>> > >> >>>>> repaired. >>>>>> > >> >>>>>>>> These >>>>>> > >> >>>>>>>>> endpoints may rely on the ability to convert JSON to >>>>>> internal >>>>>> > >> >>>>>>> protobufs. >>>>>> > >> >>>>>>>>> >>>>>> > >> >>>>>>>>> Can you clarify what you're looking to do? Are you >>>>>> looking to >>>>>> > >> >>>>>> implement >>>>>> > >> >>>>>>>> an >>>>>> > >> >>>>>>>>> API in Go that communicates with JSON instead of >>>>>> serialized >>>>>> > >> >>>>> protobuf? >>>>>> > >> >>>>>>>>> >>>>>> > >> >>>>>>>>> On Tue, Apr 8, 2014 at 1:19 PM, Vladimir Vivien >>>>>> > >> >>>>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>>> > >> >>>>>>>>> >>>>>> > >> >>>>>>>>>> Ben, >>>>>> > >> >>>>>>>>>> That is exactly what I am asking. >>>>>> > >> >>>>>>>>>> Is that something coming up soon, is there a JIRA I >>>>>> can look >>>>>> > >> >>>>> at? >>>>>> > >> >>>>>>>>>> I wanna get early start on a native json Go api or >>>>>> even help >>>>>> > >> >>>>> out if >>>>>> > >> >>>>>>>>>> possible. >>>>>> > >> >>>>>>>>>> >>>>>> > >> >>>>>>>>>> >>>>>> > >> >>>>>>>>>> On Tue, Apr 8, 2014 at 3:25 PM, Benjamin Mahler >>>>>> > >> >>>>>>>>>> <benjamin.mah...@gmail.com>wrote: >>>>>> > >> >>>>>>>>>> >>>>>> > >> >>>>>>>>>>> +vinod, benh >>>>>> > >> >>>>>>>>>>> >>>>>> > >> >>>>>>>>>>> Hey Vladimir, there will be some authenticated REST >>>>>> > >> >>>>> endpoints at >>>>>> > >> >>>>>>> some >>>>>> > >> >>>>>>>>>>> point, there is some work in this area underway. >>>>>> > >> >>>>>>>>>>> >>>>>> > >> >>>>>>>>>>> We have the ability to encode protobuf messages as >>>>>> JSON, so >>>>>> > >> >>>>> the >>>>>> > >> >>>>>>> plan >>>>>> > >> >>>>>>>>> was >>>>>> > >> >>>>>>>>>> to >>>>>> > >> >>>>>>>>>>> have any REST endpoints directly use JSON to send us >>>>>> > >> >>>>> protobuf >>>>>> > >> >>>>>>>> messages. >>>>>> > >> >>>>>>>>>> I'm >>>>>> > >> >>>>>>>>>>> not sure if this is what you're asking though? >>>>>> > >> >>>>>>>>>>> >>>>>> > >> >>>>>>>>>>> >>>>>> > >> >>>>>>>>>>> On Tue, Apr 8, 2014 at 11:13 AM, Vetoshkin Nikita < >>>>>> > >> >>>>>>>>>>> nikita.vetosh...@gmail.com> wrote: >>>>>> > >> >>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>> I'm not a mesos guy, just very curious. But in my >>>>>> opinion >>>>>> > >> >>>>> - I >>>>>> > >> >>>>>>> doubt >>>>>> > >> >>>>>>>>> it, >>>>>> > >> >>>>>>>>>>>> HTTP is synchronous request-response protocol. >>>>>> Mesos needs >>>>>> > >> >>>>>>>> something >>>>>> > >> >>>>>>>>>> more >>>>>> > >> >>>>>>>>>>>> robust for message passing. Websockets anyone? :) >>>>>> > >> >>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>> On Tue, Apr 8, 2014 at 10:08 PM, Vladimir Vivien >>>>>> > >> >>>>>>>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>>> > >> >>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>> Ben / Nikita >>>>>> > >> >>>>>>>>>>>>> Thanks for the pointers. >>>>>> > >> >>>>>>>>>>>>> So, (without digging yet) is it a fair summary to >>>>>> say >>>>>> > >> >>>>> that >>>>>> > >> >>>>>>>>> libprocess >>>>>> > >> >>>>>>>>>>>> wraps >>>>>> > >> >>>>>>>>>>>>> protobufs-encoded calls and push them over HTTP to >>>>>> > >> >>>>>>> master/slaves >>>>>> > >> >>>>>>>> ? >>>>>> > >> >>>>>>>>>> Will >>>>>> > >> >>>>>>>>>>>>> protobuf (eventually) be supplanted by direct HTTP >>>>>> via >>>>>> > >> >>>>> REST >>>>>> > >> >>>>>> or >>>>>> > >> >>>>>>>>>> similar >>>>>> > >> >>>>>>>>>>> ? >>>>>> > >> >>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>> On Mon, Apr 7, 2014 at 2:54 PM, Vetoshkin Nikita < >>>>>> > >> >>>>>>>>>>>>> nikita.vetosh...@gmail.com >>>>>> > >> >>>>>>>>>>>>>> wrote: >>>>>> > >> >>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>> Or, just to get to know - you can take tcpdump and >>>>>> > >> >>>>> take a >>>>>> > >> >>>>>>> look >>>>>> > >> >>>>>>>> :) >>>>>> > >> >>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>> I personally wouldn't call that HTTP. Something >>>>>> > >> >>>>> "HTTP-like" >>>>>> > >> >>>>>>>> would >>>>>> > >> >>>>>>>>>>>>> describe >>>>>> > >> >>>>>>>>>>>>>> it better. Because it's not request-response. It's >>>>>> > >> >>>>> just >>>>>> > >> >>>>>>> message >>>>>> > >> >>>>>>>>>>>> passing, >>>>>> > >> >>>>>>>>>>>>> no >>>>>> > >> >>>>>>>>>>>>>> need to wait for the answer - send new message one >>>>>> > >> >>>>> after >>>>>> > >> >>>>>>>> another. >>>>>> > >> >>>>>>>>>>> Every >>>>>> > >> >>>>>>>>>>>>>> message is POST with address and message type >>>>>> encoded >>>>>> > >> >>>>> in >>>>>> > >> >>>>>> URI: >>>>>> > >> >>>>>>>>> POST >>>>>> > >> >>>>>>>>>>>>>> /executor(1)/mesos.internal.RunTaskMessage. >>>>>> Sender is >>>>>> > >> >>>>>> encoded >>>>>> > >> >>>>>>>> in >>>>>> > >> >>>>>>>>>>>>> User-Agent >>>>>> > >> >>>>>>>>>>>>>> header, e.g: libprocess/slave(1)@127.0.0.1:5051. >>>>>> Body >>>>>> > >> >>>>>>> contains >>>>>> > >> >>>>>>>>>>>> protobuf >>>>>> > >> >>>>>>>>>>>>>> message, Transfer-Encoding is always "chunked". >>>>>> > >> >>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>> On Mon, Apr 7, 2014 at 10:42 PM, Benjamin Mahler >>>>>> > >> >>>>>>>>>>>>>> <benjamin.mah...@gmail.com>wrote: >>>>>> > >> >>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>> Unfortunately you will need to learn this by >>>>>> > >> >>>>> looking at >>>>>> > >> >>>>>> the >>>>>> > >> >>>>>>>>> code >>>>>> > >> >>>>>>>>>> in >>>>>> > >> >>>>>>>>>>>>>>> libprocess, as the message passing format is not >>>>>> > >> >>>>>> explicitly >>>>>> > >> >>>>>>>>>>>> documented >>>>>> > >> >>>>>>>>>>>>> at >>>>>> > >> >>>>>>>>>>>>>>> the current time. >>>>>> > >> >>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>> Start with calls like ProtobufProcess::send() and >>>>>> > >> >>>>> dig >>>>>> > >> >>>>>> your >>>>>> > >> >>>>>>>> way >>>>>> > >> >>>>>>>>>>> down. >>>>>> > >> >>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>> On Sat, Apr 5, 2014 at 7:52 AM, Vladimir Vivien >>>>>> > >> >>>>>>>>>>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>>> > >> >>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>>> I was watching this video from >>>>>> > >> >>>>>>>>>>>>>>>> >>>>>> https://www.youtube.com/watch?v=n5GT7OFSh58fromBen >>>>>> > >> >>>>>>> where >>>>>> > >> >>>>>>>> he >>>>>> > >> >>>>>>>>>>>> talked >>>>>> > >> >>>>>>>>>>>>>>>> about the wire protocol for Mesos being done in >>>>>> > >> >>>>>>>>>>>>>>>> HTTP. >>>>>> > >> >>>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>>> Where can I learn about the low-level wire >>>>>> > >> >>>>> protocol >>>>>> > >> >>>>>>> either >>>>>> > >> >>>>>>>> in >>>>>> > >> >>>>>>>>>>>>>>> documentation >>>>>> > >> >>>>>>>>>>>>>>>> or browsing through the code. >>>>>> > >> >>>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>>> Thanks. >>>>>> > >> >>>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>>> -- >>>>>> > >> >>>>>>>>>>>>>>>> Vladimir Vivien >>>>>> > >> >>>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>>> -- >>>>>> > >> >>>>>>>>>>>>> Vladimir Vivien >>>>>> > >> >>>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>>> >>>>>> > >> >>>>>>>>>>> >>>>>> > >> >>>>>>>>>> >>>>>> > >> >>>>>>>>>> >>>>>> > >> >>>>>>>>>> >>>>>> > >> >>>>>>>>>> -- >>>>>> > >> >>>>>>>>>> Vladimir Vivien >>>>>> > >> >>>>>>>>>> >>>>>> > >> >>>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>>> -- >>>>>> > >> >>>>>>>> Vladimir Vivien >>>>>> > >> >>>>>>>> >>>>>> > >> >>>>>>> >>>>>> > >> >>>>>> >>>>>> > >> >>>>> >>>>>> > >> >>>>> >>>>>> > >> >>>>> >>>>>> > >> >>>>> -- >>>>>> > >> >>>>> Vladimir Vivien >>>>>> > >> >>>>> >>>>>> > >> >>>> >>>>>> > >> >>>> >>>>>> > >> >>> >>>>>> > >> >> >>>>>> > >> >>>>>> > >> >>>>>> > > >>>>>> > > >>>>>> > > -- >>>>>> > > Vladimir Vivien >>>>>> > > >>>>>> > >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Vladimir Vivien >>>> >>> >>> >> >> >> -- >> Vladimir Vivien >> > > -- Vladimir Vivien
