Mesos / Libprocess ENETUNREACH

Hey, I’ve been testing out mesos for production recently and i’m having trouble registering frameworks over our VPN connection. I expect it’s a firewall issue, but i’m unsure as to what connectivity mesos requires for its frameworks. When running the test java framework from one of the slaves

Re: Mesos / Libprocess ENETUNREACH

-mesos-user@incubator (this mailing list is deprecated) Tom, Both the framework (and slaves) and master need to be able to talk to each other. IOW, if one of the end points uses a private IP (presumably thats the case with framework behind a VPN) then it wouldn't work. If you want the

Re: callback port

Ok got it figured out. LIBPROCESS_PORT is being unset along the way somewhere by marathon/mesos, the solution was to start the spark driver process via a script that re exports LIBPROCESS_PORT by reading it from a var named something other than LIBPROCESS_PORT (in my case SPARK_LIBPROCESS_PORT)

Q on master state.json

I see that master/state.json has state information on frameworks, where in, it has a list of all completed_tasks. Each task seems to be about 500 bytes. Does the master have a list of all completed tasks for the framework? Thinking naively about it, does it mean that if I were to run, say, 100K