Hey,
I have a Mesos cluster with a single Master. If I run the following directly on the master machine: pyspark --master mesos://host:5050 everything works just fine. If I try to connect from to the master starting a driver from my laptop everything stops after the following log output from the Spark driver: I1102 15:10:57.848831 64856064 sched.cpp:164] Version: 0.24.0 I1102 15:10:57.852708 19017728 sched.cpp:262] New master detected at master@217.66.55.19:5050 I1102 15:10:57.852934 19017728 sched.cpp:272] No credentials provided. Attempting to register without authentication The Mesos master logs show the following: I1102 15:09:50.676004 21563 master.cpp:2250] Subscribing framework Talos with checkpointing disabled and capabilities [ ] I1102 15:09:50.676686 21566 hierarchical.hpp:515] Added framework b995b024-6ec6-4bd9-b251-e4e6b1828eda-0004 I1102 15:09:51.671437 21562 http.cpp:336] HTTP GET for /master/state.json from 217.66.51.150:51588 with User-Agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36' I1102 15:09:52.357230 21567 master.cpp:2179] Received SUBSCRIBE call for framework 'Talos' at scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 I1102 15:09:52.357416 21567 master.cpp:2250] Subscribing framework Talos with checkpointing disabled and capabilities [ ] I1102 15:09:52.357520 21567 master.cpp:2260] Framework b995b024-6ec6-4bd9-b251-e4e6b1828eda-0004 (Talos) at scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 already subscribed, resending acknowledgement I1102 15:09:53.938227 21563 master.cpp:2179] Received SUBSCRIBE call for framework 'Talos' at scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 I1102 15:09:53.938426 21563 master.cpp:2250] Subscribing framework Talos with checkpointing disabled and capabilities [ ] I1102 15:09:53.938534 21563 master.cpp:2260] Framework b995b024-6ec6-4bd9-b251-e4e6b1828eda-0004 (Talos) at scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 already subscribed, resending acknowledgement I1102 15:10:00.207372 21567 master.cpp:2179] Received SUBSCRIBE call for framework 'Talos' at scheduler-fee376b3-ea04-4866-8412-957721da5a0b@10.200.120.148:51085 The framework also shows up as active in the UI and blocks all resources of the cluster, while the SUBSCRIBE calls keep coming in. Mesos authentication is completely disabled. What could be possible causes for this problem? Thanks, Sebastian ------------------------------------------------------------------------ Disclaimer The information in this email and any attachments may contain proprietary and confidential information that is intended for the addressee(s) only. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, retention or use of the contents of this information is prohibited. When addressed to our clients or vendors, any information contained in this e-mail or any attachments is subject to the terms and conditions in any governing contract. If you have received this e-mail in error, please immediately contact the sender and delete the e-mail.
