Right, this issue is related to the advertising the hostname / port in libprocess that is reachable by the Mesos Master. It should not be related to the Mesos Master running in Standalone (non HA) mode and not using Zookeeper.
-Elizabeth On Thu, Oct 22, 2015 at 1:45 PM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Thnx Rad, sounds pretty cool :). Elizabeth one note for the jira ticket i > do not run the cluster with zookeeper, i use mesos master in stand alone > mode... i guess it makes no difference right? > > On Thu, Oct 22, 2015 at 10:38 PM, Rad Gruchalski <ra...@gruchalski.com> > wrote: > >> There are 2 things: >> >> - Akka remote in 2.3.x does not support advertising hostname / port >> different to what it binds to >> - All other services: file server, broadcast server, repl class server >> do not support advertising hostnames / ports different than what they bind >> to >> >> Just to expand on the previous one, we are in the process of contributing >> the following bits: >> >> - akka-remote bind-hostname and bind-port backport to akka 2.3.x (not >> the typesafe closed support implementation, our own implementation) >> - spark patches for spark 1.4.0+ which enable running Spark on Mesos in >> Docker Bridge networking >> >> Unfortunately, due to the nature of how my employer operates I can’t >> share the code yet. We are working with our legal team to make these >> available asap. >> >> We do run this stuff in production. >> >> Kind regards, >> Radek Gruchalski >> ra...@gruchalski.com <ra...@gruchalski.com> >> de.linkedin.com/in/radgruchalski/ >> >> >> *Confidentiality:*This communication is intended for the above-named >> person and may be confidential and/or legally privileged. >> If it has come to you in error you must take no action based on it, nor >> must you copy or show it to anyone; please delete/destroy and inform the >> sender immediately. >> >> On Thursday, 22 October 2015 at 22:28, Iulian Dragoș wrote: >> >> >> >> On Thu, Oct 22, 2015 at 9:10 PM, Rad Gruchalski <ra...@gruchalski.com> >> wrote: >> >> Stavros, >> >> Spark does not support this. I am currently in the process of submitting >> patches for it however it first has to pass through the legal team at the >> company I work for. >> >> >> What exactly is missing in Spark? >> >> >> Kind regards, >> Radek Gruchalski >> ra...@gruchalski.com <ra...@gruchalski.com> >> de.linkedin.com/in/radgruchalski/ >> >> >> *Confidentiality:*This communication is intended for the above-named >> person and may be confidential and/or legally privileged. >> If it has come to you in error you must take no action based on it, nor >> must you copy or show it to anyone; please delete/destroy and inform the >> sender immediately. >> >> On Thursday, 22 October 2015 at 21:08, Stavros Kontopoulos wrote: >> >> Bridge... with the latest mesos library vesion 0.25... >> >> On Thu, Oct 22, 2015 at 9:07 PM, Elizabeth Lingg <elizab...@mesosphere.io >> > wrote: >> >> Are you using Bridge or Host Networking? >> >> -Elizabeth >> >> >> >> On Thu, Oct 22, 2015 at 12:02 PM, Stavros Kontopoulos < >> st.kontopou...@gmail.com> wrote: >> >> Hi, >> >> Im using spark on mesos on docker. I have linked my slaves to the master >> and a >> spark repl works fine inside the master container. >> >> If i try to crate the same spark repl form the host i get stuck at the >> point when the framework tries to register to the mesos master (here the >> framework is the spark repl itself). >> I can ping the container from my host and vice versa. So networking its >> not the problem. >> What i noticed form the logs is that mesos does not resolve the correct >> ip: >> >> Framework failover timeout, removing framework >> b3605c33-f573-4d40-806f-b9b0abee2e32-0012 (Spark shell) at >> scheduler-b6dbe4f8-4760-4cbe-bb5e-a991e35aba1b@127.0.1.1:40186 >> >> docker0 interface is on 172.17.x.x and my host is one such ip so i didnt >> expect there to see >> 127.0.1.1. I have tried several things like spark.driver.host, >> SPARK_LOCAL_IP to be set correctly but with no result... >> I suspect this is a mesos problem on docker... >> >> Thnx, >> >> S. >> >> >> >> >> >> >> >> -- >> >> -- >> Iulian Dragos >> >> ------ >> Reactive Apps on the JVM >> www.typesafe.com >> >> >> >
