Hello Sebastian, Did you set the MESOS_NATIVE_JAVA_LIBRARY variable before you started pyspark?
cheers, Tamas On 2 November 2015 at 15:24, Sebastian Kuepers < sebastian.kuep...@publicispixelpark.de> wrote: > Hey, > > > I have a Mesos cluster with a single Master. If I run the following > directly on the master machine: > > > pyspark --master mesos://host:5050 > > > everything works just fine. If I try to connect from to the master > starting a driver from my laptop everything stops after the following > log output from the Spark driver: > > > I1102 15:10:57.848831 64856064 sched.cpp:164] Version: 0.24.0 > I1102 15:10:57.852708 19017728 sched.cpp:262] New master detected at > master@217.66.55.19:5050 > I1102 15:10:57.852934 19017728 sched.cpp:272] No credentials provided. > Attempting to register without authentication > > > The Mesos master logs show the following: > > > I1102 15:09:50.676004 21563 master.cpp:2250] Subscribing framework Talos > with checkpointing disabled and capabilities [ ] > I1102 15:09:50.676686 21566 hierarchical.hpp:515] Added framework > b995b024-6ec6-4bd9-b251-e4e6b1828eda-0004 > I1102 15:09:51.671437 21562 http.cpp:336] HTTP GET for /master/state.json > from 217.66.51.150:51588 with User-Agent='Mozilla/5.0 (X11; Linux x86_64) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36' > I1102 15:09:52.357230 21567 master.cpp:2179] Received SUBSCRIBE call for > framework 'Talos' at > scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 > I1102 15:09:52.357416 21567 master.cpp:2250] Subscribing framework Talos > with checkpointing disabled and capabilities [ ] > I1102 15:09:52.357520 21567 master.cpp:2260] Framework > b995b024-6ec6-4bd9-b251-e4e6b1828eda-0004 (Talos) at > scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 > already subscribed, resending acknowledgement > I1102 15:09:53.938227 21563 master.cpp:2179] Received SUBSCRIBE call for > framework 'Talos' at > scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 > I1102 15:09:53.938426 21563 master.cpp:2250] Subscribing framework Talos > with checkpointing disabled and capabilities [ ] > I1102 15:09:53.938534 21563 master.cpp:2260] Framework > b995b024-6ec6-4bd9-b251-e4e6b1828eda-0004 (Talos) at > scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 > already subscribed, resending acknowledgement > I1102 15:10:00.207372 21567 master.cpp:2179] Received SUBSCRIBE call for > framework 'Talos' at > scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 > > > The framework also shows up as active in the UI and blocks all resources > of the cluster, while the SUBSCRIBE calls keep coming in. > > Mesos authentication is completely disabled. > > > What could be possible causes for this problem? > > > Thanks, > > Sebastian > > > > > ------------------------------------------------------------------------ > Disclaimer The information in this email and any attachments may contain > proprietary and confidential information that is intended for the > addressee(s) only. If you are not the intended recipient, you are hereby > notified that any disclosure, copying, distribution, retention or use of > the contents of this information is prohibited. When addressed to our > clients or vendors, any information contained in this e-mail or any > attachments is subject to the terms and conditions in any governing > contract. If you have received this e-mail in error, please immediately > contact the sender and delete the e-mail. >
