Its a custom framework that I have developed to execute long running workflows. The driver launching the framework is managed Marathon. Driver polls the database for newly submitted workflows and launch a new Mesos framework for each workflow that is submitted. Once the workflow execution finish, the SchedulerDriver is closed to stop the framework.
It will look for logs of the framework for more info. Thanks, Venkat > On Jan 11, 2017, at 2:03 PM, Greg Mann <g...@mesosphere.io> wrote: > > What framework is it? The framework should be emitting logs of its own > somewhere, we can inspect those to see what's happening framework-side. > > Cheers, > Greg > > On Wed, Jan 11, 2017 at 2:00 PM, Venkat Morampudi <venkatmoramp...@gmail.com >> wrote: > >> Hi Greg, >> >> Thanks for responding to my email. Going over the logs I found that the >> status update is being forwarded to the framework but the acknowledgement >> is not received by the master. Is there a way to enable native code logging >> to find out what going on framework side? >> >> >> >> I0111 20:10:58.663244 29820 master.cpp:5137] Status update TASK_FINISHED >> (UUID: 68dde43e-3646-4f8e-94cc-f0b07ca4b771) for task >> 324.0.0-9947ffab-b8f5-4925-94d8-12c7814ec572 of framework >> 9947ffab-b8f5-4925-94d8-12c7814ec572 from agent >> 5ee3c2ca-718f-4d48-ad38-c6757f39a833-S25 >> at slave(1)@10.92.2.29:5051 (10.92.2.29) >> I0111 20:10:58.663337 29820 master.cpp:5199] Forwarding status update >> TASK_FINISHED (UUID: 68dde43e-3646-4f8e-94cc-f0b07ca4b771) for task >> 324.0.0-9947ffab-b8f5-4925-94d8-12c7814ec572 of framework >> 9947ffab-b8f5-4925-94d8-12c7814ec572 >> I0111 20:10:58.663465 29820 master.cpp:6841] Updating the state of task >> 324.0.0-9947ffab-b8f5-4925-94d8-12c7814ec572 of framework >> 9947ffab-b8f5-4925-94d8-12c7814ec572 (latest state: TASK_FINISHED, status >> update state: TASK_FINISHED) >> >> Thanks a lot for you help, >> >> - Venkat >> >> >> >>> On Jan 11, 2017, at 12:42 PM, Greg Mann <g...@mesosphere.io> wrote: >>> >>> Venkat, >>> There should be some information in the master logs to help you debug. In >>> the case of a successful status update, you will see something like the >>> following: >>> >>> master.cpp:5848] Status update TASK_RUNNING (UUID: >>> ccb9beae-d62c-40f1-9d4e-9b6153cb0fff) for task test-task of framework >>> 8fffa9d5-bb6e-4af1-8a4a-4f5fce36df6b-0002 from agent >>> 8fffa9d5-bb6e-4af1-8a4a-4f5fce36df6b-S0 at slave(1)@127.0.0.1:5051 >>> (localhost) >>> master.cpp:5910] Forwarding status update TASK_RUNNING (UUID: >>> ccb9beae-d62c-40f1-9d4e-9b6153cb0fff) for task test-task of framework >>> 8fffa9d5-bb6e-4af1-8a4a-4f5fce36df6b-0002 >>> master.cpp:7953] Updating the state of task test-task of framework >>> 8fffa9d5-bb6e-4af1-8a4a-4f5fce36df6b-0002 (latest state: TASK_RUNNING, >>> status update state: TASK_RUNNING) >>> master.cpp:4950] Processing ACKNOWLEDGE call >>> ccb9beae-d62c-40f1-9d4e-9b6153cb0fff for task test-task of framework >>> 8fffa9d5-bb6e-4af1-8a4a-4f5fce36df6b-0002 (mesos-execute instance) on >> agent >>> 8fffa9d5-bb6e-4af1-8a4a-4f5fce36df6b-S0 >>> >>> Looking for these lines could help you discover where the issue is; i.e., >>> if the status update is being sent but not received, or if it is not >> being >>> sent to the framework at all. >>> >>> Cheers, >>> Greg >>> >>> On Wed, Jan 11, 2017 at 11:44 AM, Venkat Morampudi < >>> venkatmoramp...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> I am facing an issue with my Mesos framework with status updates. >>>> Randomly, Task Status messages doesn’t reach to the Framework scheduler. >>>> When this happens Offer declines doesn’t take affect. After sometime >> thing >>>> seem to be getting to normal. Looks like timeout is being triggered. I >>>> couldn’t find anything anything obvious from the log files. I would >> really >>>> appreciate if you can help me with this. >>>> >>>> Thanks, >>>> Venkat >> >>
