[
https://issues.apache.org/jira/browse/MESOS-9726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16823423#comment-16823423
]
Joseph Wu commented on MESOS-9726:
----------------------------------
Logs of all three mesos masters would be helpful.
Also some quick questions to get us on the same page:
* What method did you use to determine the active Mesos leader? (Marathon
leader is not the Mesos leader.)
* When you "restart" anything, is this referring to stopping/starting a
service? Or rebooting an entire node?
> No running tasks in marathon after restart non-leader mesos-master node
> -----------------------------------------------------------------------
>
> Key: MESOS-9726
> URL: https://issues.apache.org/jira/browse/MESOS-9726
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 1.4.1
> Reporter: Alexandr
> Priority: Minor
> Labels: beginner
>
> Good day!
> I have problem with my mesos-cluster:
> I have 3 mesos-master (Mesos version 1.4.1) nodes with Marathon cluster v
> 1.5.11 (lets call them - Mas1,Mas2,Mas3) and 3 mesos-slave nodes (lets call
> them - Slv1,Slv2,Slv3) with running apps on it. And I also have a Zookeeper
> cluster on nodes Mas1,Mas2,Mas3.
> So, Mesos leader - master-node "Mas1".
> After I restarted master-node "Mas3" - he got back to the Mesos cluster,
> everything is fine but after a moment I opened Marathon and all running tasks
> from my mesos-slave nodes became "unknown" and have no instances on it.
> So I checked:
> 1. My mesos agents - everything was ok, 3 agents running.
> 2. That all services are running and all clusters (Mesos\Marathon\Zookeeper)
> are fine
> 3. Decided to restart all mesos-slave services on slave nodes - on
> slave-node Slv3 1 of 3 instances launched for all applications, then
> restarted all marathon-services. After it all tasks switched to status
> "Waiting"\"Delayed".
> 4. Checked mesos-master and slave logs, no errors or information about any
> problems on cluster - only information about killing and launching new tasks
> on slave-node.
> 5. Decided to stop-start mesos-master service for re-election of a Mesos
> leader.
> After it leader became master-node "Mas2" and all tasks in marathon started
> to run instances like normal.
> Logs will be uploaded later. Wonder how it could happen
> {code:java}
> Apr 10 18:32:18 Mas1 docker[*****]: I0410 15:32:18.342406 12 http.cpp:**]
> HTTP GET for /master/state from ****:*** with User-Agent='Go-http-client/1.1'
> Apr 10 18:32:18 Mas1 docker[*****]: I0410 15:32:18.391480 9
> http.cpp:1185] HTTP GET for /master/state from *****:40686 with
> User-Agent='Go-http-client/1.1' Apr 10 18:32:18 Mas1 docker[27956]: E0410
> 15:32:18.567880 14 process.cpp:2577] Failed to shutdown socket with fd 44,
> address ****:5050: Transport endpoint is not connected Apr 10 18:32:18 Mas1
> docker[27956]: E0410 15:32:18.596901 14 process.cpp:2577] Failed to
> shutdown socket with fd 46, address ****:5050: Transport endpoint is not
> connected
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
