Re: Can health-checks be run by Mesos for docker tasks?

[Jay Taylor]

I am using my own framework, and the full task info I'm using is posted earlier 
in this thread.  Do you happen to know if Marathon uses Mesos's health checks 
for its health check system?



> On Oct 6, 2015, at 9:01 PM, haosdent  wrote:
> 
> Yes, launch the health task through its definition in taskinfo. Do you launch 
> your task through Marathon? I could test it in my side.
> 
>> On Wed, Oct 7, 2015 at 11:56 AM, Jay Taylor  wrote:
>> Precisely, and there are none of those statements.  Are you or others 
>> confident health-checks are part of the code path when defined via task info 
>> for docker container tasks?  Going through the code, I wasn't able to find 
>> the linkage for anything other than health-checks triggered through a custom 
>> executor.
>> 
>> With that being said it is a pretty good sized code base and I'm not very 
>> familiar with it, so my analysis this far has by no means been exhaustive.
>> 
>> 
>> 
>>> On Oct 6, 2015, at 8:41 PM, haosdent  wrote:
>>> 
>>> When health check launch, it would have a log like this in your executor 
>>> stdout
>>> ```
>>> Health check process launched at pid xxx
>>> ```
>>> 
 On Wed, Oct 7, 2015 at 11:37 AM, Jay Taylor  wrote:
 I'm happy to try this, however wouldn't there be output in the logs with 
 the string "health" or "Health" if the health-check were active?  None of 
 my master or slave logs contain the string..
 
 
 
> On Oct 6, 2015, at 7:45 PM, haosdent  wrote:
> 
> Could you use "exit 1" instead of "sleep 5" to see whether could see 
> unhealthy status in your task stdout/stderr.
> 
>> On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor  wrote:
>> My current version is 0.24.1.
>> 
>>> On Tue, Oct 6, 2015 at 7:30 PM, haosdent  wrote:
>>> yes, adam also help commit it to 0.23.1 and 0.24.1 
>>> https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0
>>>  
>>> https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7
>>> Are you use one of this version?
>>> 
 On Wed, Oct 7, 2015 at 10:26 AM, haosdent  wrote:
 I remember 0.23.1 and 0.24.1 contains this backport, let me double 
 check.
 
> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor  
> wrote:
> Oops- Now I see you already said it's in master.  I'll look there :)
> 
> Thanks again!
> 
>> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  
>> wrote:
>> Great, thanks for the quick reply Tim!
>> 
>> Do you know if there is a branch I can checkout to test it out?
>> 
>>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen  
>>> wrote:
>>> Hi Jay, 
>>> 
>>> We just added health check support for docker tasks that's in 
>>> master but not yet released. It will run docker exec with the 
>>> command you provided as health checks.
>>> 
>>> It should be in the next release.
>>> 
>>> Thanks!
>>> 
>>> Tim
>>> 
>>> 
 On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
 
 Does Mesos support health checks for docker image tasks?  Mesos 
 seems to be ignoring the TaskInfo.HealthCheck field for me.
 
 Example TaskInfo JSON received back from Mesos:
 
>> {
>>   "name":"hello-app.web.v3",
>>   "task_id":{
>> 
>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>>   },
>>   "slave_id":{
>> "value":"20150924-210922-1608624320-5050-1792-S1"
>>   },
>>   "resources":[
>> {
>>   "name":"cpus",
>>   "type":0,
>>   "scalar":{
>> "value":0.1
>>   }
>> },
>> {
>>   "name":"mem",
>>   "type":0,
>>   "scalar":{
>> "value":256
>>   }
>> },
>> {
>>   "name":"ports",
>>   "type":1,
>>   "ranges":{
>> "range":[
>>   {
>> "begin":31002,
>> "end":31002
>>   }
>> ]
>>   }
>> }
>>   ],
>>   "command":{
>> "container":{
>>   
>> "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>> 

Re: Broken images in documentation on web

[haosdent]

Hi, @Brian. Joseph have a patch for this MESOS-3183
, but the website still
not update. Before the document in website update, you could read these
documents in github mirror https://github.com/apache/mesos/tree/master/docs
first. Thank you for your report

On Tue, Oct 6, 2015 at 10:14 PM, Brian Candler  wrote:

> It seems several documentation pages now have broken images, for example:
>
> http://mesos.apache.org/documentation/latest/external-containerizer/
> (under " Container Lifecycle Sequence Diagrams")
>
> http://mesos.apache.org/documentation/latest/oversubscription/
> (under "How does it work?")
>
> However, some pages are OK, e.g.
> http://mesos.apache.org/documentation/latest/mesos-architecture/
>
> Could someone investigate please?
>
> Thanks,
>
> Brian Candler.
>
>


-- 
Best Regards,
Haosdent Huang

Re: Can health-checks be run by Mesos for docker tasks?

[Jay Taylor]

I'm happy to try this, however wouldn't there be output in the logs with the 
string "health" or "Health" if the health-check were active?  None of my master 
or slave logs contain the string..



> On Oct 6, 2015, at 7:45 PM, haosdent  wrote:
> 
> Could you use "exit 1" instead of "sleep 5" to see whether could see 
> unhealthy status in your task stdout/stderr.
> 
>> On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor  wrote:
>> My current version is 0.24.1.
>> 
>>> On Tue, Oct 6, 2015 at 7:30 PM, haosdent  wrote:
>>> yes, adam also help commit it to 0.23.1 and 0.24.1 
>>> https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0
>>>  
>>> https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7
>>> Are you use one of this version?
>>> 
 On Wed, Oct 7, 2015 at 10:26 AM, haosdent  wrote:
 I remember 0.23.1 and 0.24.1 contains this backport, let me double check.
 
> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor  wrote:
> Oops- Now I see you already said it's in master.  I'll look there :)
> 
> Thanks again!
> 
>> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:
>> Great, thanks for the quick reply Tim!
>> 
>> Do you know if there is a branch I can checkout to test it out?
>> 
>>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen  wrote:
>>> Hi Jay, 
>>> 
>>> We just added health check support for docker tasks that's in master 
>>> but not yet released. It will run docker exec with the command you 
>>> provided as health checks.
>>> 
>>> It should be in the next release.
>>> 
>>> Thanks!
>>> 
>>> Tim
>>> 
>>> 
 On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
 
 Does Mesos support health checks for docker image tasks?  Mesos seems 
 to be ignoring the TaskInfo.HealthCheck field for me.
 
 Example TaskInfo JSON received back from Mesos:
 
>> {
>>   "name":"hello-app.web.v3",
>>   "task_id":{
>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>>   },
>>   "slave_id":{
>> "value":"20150924-210922-1608624320-5050-1792-S1"
>>   },
>>   "resources":[
>> {
>>   "name":"cpus",
>>   "type":0,
>>   "scalar":{
>> "value":0.1
>>   }
>> },
>> {
>>   "name":"mem",
>>   "type":0,
>>   "scalar":{
>> "value":256
>>   }
>> },
>> {
>>   "name":"ports",
>>   "type":1,
>>   "ranges":{
>> "range":[
>>   {
>> "begin":31002,
>> "end":31002
>>   }
>> ]
>>   }
>> }
>>   ],
>>   "command":{
>> "container":{
>>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>> },
>> "shell":false
>>   },
>>   "container":{
>> "type":1,
>> "docker":{
>>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>>   "network":2,
>>   "port_mappings":[
>> {
>>   "host_port":31002,
>>   "container_port":8000,
>>   "protocol":"tcp"
>> }
>>   ],
>>   "privileged":false,
>>   "parameters":[],
>>   "force_pull_image":false
>> }
>>   },
>>   "health_check":{
>> "delay_seconds":5,
>> "interval_seconds":10,
>> "timeout_seconds":10,
>> "consecutive_failures":3,
>> "grace_period_seconds":0,
>> "command":{
>>   "shell":true,
>>   "value":"sleep 5",
>>   "user":"root"
>> }
>>   }
>> }
 
 I have searched all machines and containers to see if they ever run 
 the command (in this case `sleep 5`), but have not found any 
 indication that it is being executed.
 
 In the mesos src code the health-checks are invoked from 
 src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does 
 this mean that health-checks are only supported for custom executors 
 and not for docker tasks?
 
 What I am trying to accomplish is to have the 0/non-zero exit-status 
 of a health-check command translate to task health.
 
 Thanks!
 Jay
 
 
 
 -- 
 Best Regards,
 Haosdent Huang
>>> 
>>> 
>>> 
>>> -- 
>>> 

Re: Can health-checks be run by Mesos for docker tasks?

[haosdent]

When health check launch, it would have a log like this in your executor
stdout
```
Health check process launched at pid xxx
```

On Wed, Oct 7, 2015 at 11:37 AM, Jay Taylor  wrote:

> I'm happy to try this, however wouldn't there be output in the logs with
> the string "health" or "Health" if the health-check were active?  None of
> my master or slave logs contain the string..
>
>
>
> On Oct 6, 2015, at 7:45 PM, haosdent  wrote:
>
> Could you use "exit 1" instead of "sleep 5" to see whether could see
> unhealthy status in your task stdout/stderr.
>
> On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor  wrote:
>
>> My current version is 0.24.1.
>>
>> On Tue, Oct 6, 2015 at 7:30 PM, haosdent  wrote:
>>
>>> yes, adam also help commit it to 0.23.1 and 0.24.1
>>> https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0
>>>
>>> https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7
>>> Are you use one of this version?
>>>
>>> On Wed, Oct 7, 2015 at 10:26 AM, haosdent  wrote:
>>>
 I remember 0.23.1 and 0.24.1 contains this backport, let me double
 check.

 On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor 
 wrote:

> Oops- Now I see you already said it's in master.  I'll look there :)
>
> Thanks again!
>
> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:
>
>> Great, thanks for the quick reply Tim!
>>
>> Do you know if there is a branch I can checkout to test it out?
>>
>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen 
>> wrote:
>>
>>> Hi Jay,
>>>
>>> We just added health check support for docker tasks that's in master
>>> but not yet released. It will run docker exec with the command you 
>>> provided
>>> as health checks.
>>>
>>> It should be in the next release.
>>>
>>> Thanks!
>>>
>>> Tim
>>>
>>>
>>> On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
>>>
>>> Does Mesos support health checks for docker image tasks?  Mesos
>>> seems to be ignoring the TaskInfo.HealthCheck field for me.
>>>
>>> Example TaskInfo JSON received back from Mesos:
>>>
>>> {

   "name":"hello-app.web.v3",

   "task_id":{

 "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"

   },

   "slave_id":{

 "value":"20150924-210922-1608624320-5050-1792-S1"

   },

   "resources":[

 {

   "name":"cpus",

   "type":0,

   "scalar":{

 "value":0.1

   }

 },

 {

   "name":"mem",

   "type":0,

   "scalar":{

 "value":256

   }

 },

 {

   "name":"ports",

   "type":1,

   "ranges":{

 "range":[

   {

 "begin":31002,

 "end":31002

   }

 ]

   }

 }

   ],

   "command":{

 "container":{

   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"

 },

 "shell":false

   },

   "container":{

 "type":1,

 "docker":{

   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",

   "network":2,

   "port_mappings":[

 {

   "host_port":31002,

   "container_port":8000,

   "protocol":"tcp"

 }

   ],

   "privileged":false,

   "parameters":[],

   "force_pull_image":false

 }

   },

   "health_check":{

 "delay_seconds":5,

 "interval_seconds":10,

 "timeout_seconds":10,

 "consecutive_failures":3,

 "grace_period_seconds":0,

 "command":{

   "shell":true,

   "value":"sleep 5",

   "user":"root"

 }

   }

 }


>>> I have searched 

Re: Can health-checks be run by Mesos for docker tasks?

[Jay Taylor]

Precisely, and there are none of those statements.  Are you or others confident 
health-checks are part of the code path when defined via task info for docker 
container tasks?  Going through the code, I wasn't able to find the linkage for 
anything other than health-checks triggered through a custom executor.

With that being said it is a pretty good sized code base and I'm not very 
familiar with it, so my analysis this far has by no means been exhaustive.



> On Oct 6, 2015, at 8:41 PM, haosdent  wrote:
> 
> When health check launch, it would have a log like this in your executor 
> stdout
> ```
> Health check process launched at pid xxx
> ```
> 
>> On Wed, Oct 7, 2015 at 11:37 AM, Jay Taylor  wrote:
>> I'm happy to try this, however wouldn't there be output in the logs with the 
>> string "health" or "Health" if the health-check were active?  None of my 
>> master or slave logs contain the string..
>> 
>> 
>> 
>>> On Oct 6, 2015, at 7:45 PM, haosdent  wrote:
>>> 
>>> Could you use "exit 1" instead of "sleep 5" to see whether could see 
>>> unhealthy status in your task stdout/stderr.
>>> 
 On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor  wrote:
 My current version is 0.24.1.
 
> On Tue, Oct 6, 2015 at 7:30 PM, haosdent  wrote:
> yes, adam also help commit it to 0.23.1 and 0.24.1 
> https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0
>  
> https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7
> Are you use one of this version?
> 
>> On Wed, Oct 7, 2015 at 10:26 AM, haosdent  wrote:
>> I remember 0.23.1 and 0.24.1 contains this backport, let me double check.
>> 
>>> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor  wrote:
>>> Oops- Now I see you already said it's in master.  I'll look there :)
>>> 
>>> Thanks again!
>>> 
 On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:
 Great, thanks for the quick reply Tim!
 
 Do you know if there is a branch I can checkout to test it out?
 
> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen  
> wrote:
> Hi Jay, 
> 
> We just added health check support for docker tasks that's in master 
> but not yet released. It will run docker exec with the command you 
> provided as health checks.
> 
> It should be in the next release.
> 
> Thanks!
> 
> Tim
> 
> 
>> On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
>> 
>> Does Mesos support health checks for docker image tasks?  Mesos 
>> seems to be ignoring the TaskInfo.HealthCheck field for me.
>> 
>> Example TaskInfo JSON received back from Mesos:
>> 
 {
   "name":"hello-app.web.v3",
   "task_id":{
 "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
   },
   "slave_id":{
 "value":"20150924-210922-1608624320-5050-1792-S1"
   },
   "resources":[
 {
   "name":"cpus",
   "type":0,
   "scalar":{
 "value":0.1
   }
 },
 {
   "name":"mem",
   "type":0,
   "scalar":{
 "value":256
   }
 },
 {
   "name":"ports",
   "type":1,
   "ranges":{
 "range":[
   {
 "begin":31002,
 "end":31002
   }
 ]
   }
 }
   ],
   "command":{
 "container":{
   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
 },
 "shell":false
   },
   "container":{
 "type":1,
 "docker":{
   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
   "network":2,
   "port_mappings":[
 {
   "host_port":31002,
   "container_port":8000,
   "protocol":"tcp"
 }
   ],
   "privileged":false,
   "parameters":[],
   "force_pull_image":false
 }
   },
   "health_check":{
 "delay_seconds":5,
 "interval_seconds":10,
 "timeout_seconds":10,

Re: Can health-checks be run by Mesos for docker tasks?

[haosdent]

Yes, launch the health task through its definition in taskinfo. Do you
launch your task through Marathon? I could test it in my side.

On Wed, Oct 7, 2015 at 11:56 AM, Jay Taylor  wrote:

> Precisely, and there are none of those statements.  Are you or others
> confident health-checks are part of the code path when defined via task
> info for docker container tasks?  Going through the code, I wasn't able to
> find the linkage for anything other than health-checks triggered through a
> custom executor.
>
> With that being said it is a pretty good sized code base and I'm not very
> familiar with it, so my analysis this far has by no means been exhaustive.
>
>
>
> On Oct 6, 2015, at 8:41 PM, haosdent  wrote:
>
> When health check launch, it would have a log like this in your executor
> stdout
> ```
> Health check process launched at pid xxx
> ```
>
> On Wed, Oct 7, 2015 at 11:37 AM, Jay Taylor  wrote:
>
>> I'm happy to try this, however wouldn't there be output in the logs with
>> the string "health" or "Health" if the health-check were active?  None of
>> my master or slave logs contain the string..
>>
>>
>>
>> On Oct 6, 2015, at 7:45 PM, haosdent  wrote:
>>
>> Could you use "exit 1" instead of "sleep 5" to see whether could see
>> unhealthy status in your task stdout/stderr.
>>
>> On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor  wrote:
>>
>>> My current version is 0.24.1.
>>>
>>> On Tue, Oct 6, 2015 at 7:30 PM, haosdent  wrote:
>>>
 yes, adam also help commit it to 0.23.1 and 0.24.1
 https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0

 https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7
 Are you use one of this version?

 On Wed, Oct 7, 2015 at 10:26 AM, haosdent  wrote:

> I remember 0.23.1 and 0.24.1 contains this backport, let me double
> check.
>
> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor 
> wrote:
>
>> Oops- Now I see you already said it's in master.  I'll look there :)
>>
>> Thanks again!
>>
>> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:
>>
>>> Great, thanks for the quick reply Tim!
>>>
>>> Do you know if there is a branch I can checkout to test it out?
>>>
>>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen 
>>> wrote:
>>>
 Hi Jay,

 We just added health check support for docker tasks that's in
 master but not yet released. It will run docker exec with the command 
 you
 provided as health checks.

 It should be in the next release.

 Thanks!

 Tim


 On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:

 Does Mesos support health checks for docker image tasks?  Mesos
 seems to be ignoring the TaskInfo.HealthCheck field for me.

 Example TaskInfo JSON received back from Mesos:

 {
>
>   "name":"hello-app.web.v3",
>
>   "task_id":{
>
> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>
>   },
>
>   "slave_id":{
>
> "value":"20150924-210922-1608624320-5050-1792-S1"
>
>   },
>
>   "resources":[
>
> {
>
>   "name":"cpus",
>
>   "type":0,
>
>   "scalar":{
>
> "value":0.1
>
>   }
>
> },
>
> {
>
>   "name":"mem",
>
>   "type":0,
>
>   "scalar":{
>
> "value":256
>
>   }
>
> },
>
> {
>
>   "name":"ports",
>
>   "type":1,
>
>   "ranges":{
>
> "range":[
>
>   {
>
> "begin":31002,
>
> "end":31002
>
>   }
>
> ]
>
>   }
>
> }
>
>   ],
>
>   "command":{
>
> "container":{
>
>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>
> },
>
> "shell":false
>
>   },
>
>   "container":{
>
> "type":1,
>
> "docker":{
>
>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>
>   "network":2,
>
>   

Re: MesosCon EU Hackathon hot list?

[Casey Bisson]

Joyent has just announced additional details of their sponsorship, including 
prizes:

http://www.joyent.com/blog/getting-ready-for-mesoscon-hackathon

• Joyent is offering every participant a $10,000 credit for Triton Elastic 
Container services

• Each member of the winning teams in the community need and people’s choice 
categories will receive an Apple Watch Sport

• Winners of Joyent's container-native bonus awards will receive an additional 
$10,000 credit1 for Triton Elastic Container services. Again, this award goes 
to each team member



Fine print:

• Credits may only be used for Triton services in the Joyent public cloud. 
Credits are non-transferable and expire October 31, 2016. Maximum monthly usage 
limited to $1,000. Recipient must accept Joyent's standard terms of service and 
register their user information with Joyent staff during the event.

• Up to four participants per team, per hackathon rules.



Requests to the community:

• Please be sure to register for MesosCon Europe and tick the box saying you 
want to join the hackathon

• If you're an experienced Mesos hacker, please consider offering to mentor a 
project. Announce your availability via the mail list or on Twitter.

• Use the hashtag #mesohack in tweets and on Instagram and elsewhere



Finally, we can't wait to see your hacks, and will be toasting the winners at 
an After-Hack party Friday evening at the Habourmaster Pub. See you in Dublin!




> On Oct 5, 2015, at 2:15 PM, Casey Bisson  wrote:
> 
> The MesosCon EU hackathon is this Friday!
> 
> Hopefully we all get a chance to build something silly, or fantastic, or just 
> work through a really frustrating bug, but  this is a hackathon, so the goal 
> is to meet people, have fun, and learn from each other. 
> 
> Joyent is sponsoring the event and offering some awards in categories 
> outlined in https://www.joyent.com/blog/mesos-we-heart-hack-you: 
> 
> 
>> Community need: These are issues logged in the Mesos issue tracker that have 
>> received a large amount of activity (comments, votes, etc), and address some 
>> of the larger problems or pain points for the community. Suggested projects 
>> include new features, bugs, or documentation.
>> 
>> People’s choice: These contributions will be voted on by your peers, and our 
>> guidance is intentionally open-ended so teams will run wild with their 
>> ideas. We’re excited to see what you come up with.
> 
> I wanted to start conversation about that first category, the community 
> needs. Is there a good reference for suggestions about those needs? Are there 
> any teams forming around them?
> 
> Additionally, I’m wondering who will be on site for the hackathon. Are their 
> people who are willing to be mentors? Are there others who are looking for 
> mentors?
> 
> I’m looking forward to a fantastic event and meeting many people in the 
> community. Thank you,
> 
> —Casey*
> 
> * Oh, hai. This is my first post to these lists. I work for Joyent on Triton, 
> the container-native data center automation solution for public and private 
> clouds. Learn more at https://www.joyent.com/triton 
> .

mesos framework authentication: authentication failure

[Harry Metske]

Hi,

I am trying to setup mesos framework authentication.
I created the necessary config files and now mesos starts up with the
--authenticate="true"  flag :

I1006 20:11:52.940199  7866 master.cpp:380] Flags at startup:
--allocation_interval="1secs" --allocator="HierarchicalDRF"
--authenticate="true" --authenticate_slaves="true"
--authenticators="crammd5" --authorizers="local" --cluster="cluster1"
--credentials="/etc/mesos-master-credentials" --framework_sorter="drf"
--help="false" --initialize_driver_logging="true"
--log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
--logging_level="INFO" --max_slave_ping_timeouts="5" --port="5050"
--quiet="false" --quorum="1" --recovery_slave_removal_limit="100%"
--registry="replicated_log" --registry_fetch_timeout="1mins"
--registry_store_timeout="5secs" --registry_strict="false"
--root_submissions="true" --slave_ping_timeout="15secs"
--slave_reregister_timeout="10mins" --user_sorter="drf" --version="false"
--webui_dir="/usr/share/mesos/webui" --work_dir="/var/lib/mesos"
--zk="zk://localhost:2181/mesos" --zk_session_timeout="10secs"

The file /etc/mesos-master-credentials contains 2 lines, one for slaves
(slave-authentication already works fine), and one line contains the user
and password of marathon:

root@mm2:/var/log/mesos# cat /etc/mesos-master-credentials
marathon-user marathon-passw0rd
slave-user slave-passw0rd

Then I startup marathon and see it has the flags
--mesos_authentication_secret_file and --mesos_authentication_principal :

root@mm2:/var/log/mesos# service marathon start
Starting marathon: run_jar --mesos_authentication_secret_file
/etc/marathon/mesos_authentication_secret --http_credentials admini:adm1n1
--mesos_authentication_principal marathon-user --ssl_keystore_password
notsecret --ssl_keystore_path /etc/marathon/marathon-keystore.jks
--mesos_user taskuser --zk zk://localhost:2181/marathon --master
zk://localhost:2181/mesos

The mesos_authentication_secret_file contains the proper password (not
trailing LF or CR):

root@mm2:/var/log/mesos# cat /etc/marathon/mesos_authentication_secret
marathon-passw0rd

But authentication fails :

The mesos log  shows:

I1006 20:14:55.146001  7886 master.cpp:4694] Authenticating
scheduler-d7d6b45a-fd3d-4e96-bca7-c7974e5dccd2@172.17.0.1:39104
I1006 20:14:55.146437  7886 authenticator.cpp:92] Creating new server SASL
connection
I1006 20:14:55.148047  7884 authenticator.cpp:197] Received SASL
authentication start
I1006 20:14:55.148223  7884 authenticator.cpp:319] Authentication requires
more steps
I1006 20:14:55.149839  7883 authenticator.cpp:225] Received SASL
authentication step
W1006 20:14:55.150003  7883 authenticator.cpp:325] Authentication failure:
authentication failure
W1006 20:14:55.150168  7883 master.cpp:4721] Failed to authenticate
scheduler-d7d6b45a-fd3d-4e96-bca7-c7974e5dccd2@172.17.0.1:39104: Refused
authentication


The marathon log shows :

Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.137357  8146
sched.cpp:318] Authenticating with master master@172.17.0.1:5050
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.137375  8146
sched.cpp:325] Using default CRAM-MD5 authenticatee
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.137688  8146
authenticatee.cpp:91] Initializing client SASL
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.145184  8146
authenticatee.cpp:115] Creating new client SASL connection
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.147569  8143
authenticatee.cpp:206] Received SASL authentication mechanisms: CRAM-MD5
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.147620  8143
authenticatee.cpp:232] Attempting to authenticate with mechanism 'CRAM-MD5'
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.148651  8143
authenticatee.cpp:252] Received SASL authentication step
Oct  6 18:14:55 mm2 marathon[8071]: E1006 18:14:55.150774  8143
sched.cpp:401] Master master@172.17.0.1:5050 refused authentication
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.150810  8143
sched.cpp:1016] Got error 'Master refused authentication'
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.150822  8143
sched.cpp:1782] Asked to abort the driver
Oct  6 18:14:55 mm2 marathon[8071]: [INFO] [10/06/2015 18:14:55.153]
[marathon-akka.actor.default-dispatcher-2]
[akka://marathon/user/MarathonScheduler/$a] Scheduler actor ready
Oct  6 18:14:55 mm2 marathon[8071]: [2015-10-06 18:14:55,155] WARN Error:
Master refused authentication
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$456204a5:113)
Oct  6 18:14:55 mm2 marathon[8071]: [2015-10-06 18:14:55,156] FATAL
Committing suicide!
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$456204a5:134)
Oct  6 18:14:55 mm2 marathon[8071]: I1006 18:14:55.299145  8143
sched.cpp:1062] Aborting framework ''

My environment:
* Ubuntu 14.04.03 LTS
* Oracle Java 1.8.0_60-b27
* Mesos 0.24.1
* Marathon 0.11.0

Am I missing something here?
What can be the reason for this failure?
How can I further debug this issue ?

kind regards,
Harry

[VOTE] Release Apache Mesos 0.25.0 (rc2)

[Niklas Nielsen]

Hi all,

Please vote on releasing the following candidate as Apache Mesos 0.25.0.



0.25.0 includes the following:



 * [MESOS-1474] - Experimental support for maintenance primitives.

 * [MESOS-2600] - Added master endpoints /reserve and /unreserve for
dynamic reservations.

 * [MESOS-2044] - Extended Module APIs to enable IP per container
assignment, isolation and resolution.


** Bug fixes

  * [MESOS-2635] - Web UI Display Bug when starting lots of tasks with
small cpu value.

  * [MESOS-2986] - Docker version output is not compatible with Mesos.

  * [MESOS-3046] - Stout's UUID re-seeds a new random generator during each
call to UUID::random.

  * [MESOS-3051] - performance issues with port ranges comparison.

  * [MESOS-3052] - Allocator performance issue when using a large number of
filters.

  * [MESOS-3136] - COMMAND health checks with Marathon 0.10.0 are broken.

  * [MESOS-3169] - FrameworkInfo should only be updated if the
re-registration is valid.

  * [MESOS-3185] - Refactor Subprocess logic in linux/perf.cpp to use
common subroutine.

  * [MESOS-3239] - Refactor master HTTP endpoints help messages such that
they cannot be out of sync.

  * [MESOS-3245] - The comments of DRFSorter::dirty is not correct.

  * [MESOS-3254] - Cgroup CHECK fails test harness.

  * [MESOS-3258] - Remove Frameworkinfo capabilities on re-registration.

  * [MESOS-3261] - Move QoS plug-ins to a specified folder like
resource_estimator.

  * [MESOS-3269] - The comments of Master::updateSlave() is not correct.

  * [MESOS-3282] - Web UI no longer shows Tasks information.

  * [MESOS-3344] - Add more comments for strings::internal::fmt.

  * [MESOS-3351] - duplicated slave id in master after master failover.

  * [MESOS-3387] - Refactor MesosContainerizer to accept namespace
dynamically.

  * [MESOS-3408] - Labels field of FrameworkInfo should be added into v1
mesos.proto.

  * [MESOS-3411] - ReservationEndpointsTest.AvailableResources appears to
be faulty.

  * [MESOS-3423] - Perf event isolator stops performing sampling if a
single timeout occurs.

  * [MESOS-3426] - process::collect and process::await do not perform
discard propagation.

  * [MESOS-3430] -
LinuxFilesystemIsolatorTest.ROOT_PersistentVolumeWithoutRootFilesystem
fails on CentOS 7.1.

  * [MESOS-3450] - Update Mesos C++ Style Guide for namespace usage.

  * [MESOS-3451] - Failing tests after changes to
Isolator/MesosContainerizer API.

  * [MESOS-3458] - Segfault when accepting or declining inverse offers.

  * [MESOS-3474] - ExamplesTest.{TestFramework, JavaFramework,
PythonFramework} failed on CentOS 6.

  * [MESOS-3489] - Add support for exposing Accept/Decline responses for
inverse offers.

  * [MESOS-3490] - Mesos UI fails to represent JSON entities.

  * [MESOS-3512] - Don't retry close() on EINTR.

  * [MESOS-3513] - Cgroups Test Filters aborts tests on Centos 6.6.

  * [MESOS-3519] - Fix file descriptor leakage / double close in the code
base.

  * [MESOS-3538] -
CgroupsNoHierarchyTest.ROOT_CGROUPS_NOHIERARCHY_MountUnmountHierarchy test
is flaky.

  * [MESOS-3575] - V1 API java/python protos are not generated.


** Improvements

  * [MESOS-2719] - Deprecating '.json' extension in master endpoints urls.

  * [MESOS-2757] - Add -> operator for Option, Try, Result,
Future.

  * [MESOS-2875] - Add containerId to ResourceUsage to enable QoS
controller to target a container.

  * [MESOS-2964] - libprocess io does not support peek().

  * [MESOS-2983] - Deprecating '.json' extension in slave endpoints url.

  * [MESOS-2984] - Deprecating '.json' extension in files endpoints url.

  * [MESOS-3037] - Add a SUPPRESS call to the scheduler.

  * [MESOS-3187] - Docker cli option support.

  * [MESOS-3304] - Remove remnants of LIBPROCESS_STATISTICS_WINDOW.

  * [MESOS-3312] - Factor out JSON to repeated protobuf conversion.

  * [MESOS-3340] - Command-line flags should take precedence over OS Env
variables.

  * [MESOS-3347] - Remove dead code in src/linux/perf.cpp.

  * [MESOS-3377] - mesos docker container with container_name as ENV
variable.

  * [MESOS-3457] - Add flag to disable hostname lookup.


The full CHANGELOG for the release is available at:

https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.25.0-rc2




The candidate for Mesos 0.25.0 release is available at:

https://dist.apache.org/repos/dist/dev/mesos/0.25.0-rc2/mesos-0.25.0.tar.gz


The tag to be voted on is 0.25.0-rc2:

https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.25.0-rc2


The MD5 checksum of the tarball can be found at:

https://dist.apache.org/repos/dist/dev/mesos/0.25.0-rc2/mesos-0.25.0.tar.gz.md5


The signature of the tarball can be found at:

https://dist.apache.org/repos/dist/dev/mesos/0.25.0-rc2/mesos-0.25.0.tar.gz.asc


The PGP key used to sign the release is here:

Re: Scheduling tasks based on dependancy

[Pradeep Kiruvale]

Hi Sharma,

Awesome!. This is what I was looking for. Thanks for the reply.

I will have a look for more info in wiki.

Regards,
Pradeep

On 5 October 2015 at 18:28, Sharma Podila  wrote:

> Pradeep,
>
> We recently open sourced Fenzo  (wiki
> ) to handle these scenarios. We
> add a custom attribute for network bandwidth for each agent's "mesos-slave"
> command line. And we have Fenzo assign resources to tasks based on CPU,
> memory, disk, ports, and network bandwidth requirements. With Fenzo you can
> define affinity, locality, and any other custom scheduling objectives using
> plugins. Some of the plugins are already built in. It is also easy to add
> additional plugins to cover other objectives you care about.
>
> "Dependencies" can mean multiple things. Do you mean dependencies on
> certain attributes of resources/agents? Dependencies on where other tasks
> are assigned? All of these are covered. However, if you mean workflow type
> of dependencies on completion of other tasks, then, there are no built in
> plugins. You could write one using Fenzo. It is also common for such
> workflow dependencies to be covered by an entity external to the scheduler.
> Both techniques can be made to work.
>
> Fenzo has the concept of hard Vs soft constraints. You could specify, for
> example, resource affinity and/or task locality as a soft constraint or a
> hard constraint. See the wiki docs link I provided above for details.
>
> Sharma
>
>
> On Mon, Oct 5, 2015 at 8:21 AM, Pradeep Kiruvale <
> pradeepkiruv...@gmail.com> wrote:
>
>> Hi All,
>>
>> Are there any frameworks that exists with the Mesos to schedule the
>> bigger apps?
>> I mean to say scheduling a app which has many services and will not fit
>> into one physical node.
>>
>> Is there any frame work that can be used to
>>  schedule tasks based on the underlying hardware constraints like Network
>> bandwidth ?
>>  Schedule the tasks based on their dependencies and proximity to each
>> other in a cluster or a rack?
>>
>> Thanks & Regards,
>> Pradeep
>>
>
>

Can health-checks be run by Mesos for docker tasks?

[Jay Taylor]

Does Mesos support health checks for docker image tasks?  Mesos seems to be
ignoring the TaskInfo.HealthCheck field for me.

Example TaskInfo JSON received back from Mesos:

{
>
>   "name":"hello-app.web.v3",
>
>   "task_id":{
>
> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>
>   },
>
>   "slave_id":{
>
> "value":"20150924-210922-1608624320-5050-1792-S1"
>
>   },
>
>   "resources":[
>
> {
>
>   "name":"cpus",
>
>   "type":0,
>
>   "scalar":{
>
> "value":0.1
>
>   }
>
> },
>
> {
>
>   "name":"mem",
>
>   "type":0,
>
>   "scalar":{
>
> "value":256
>
>   }
>
> },
>
> {
>
>   "name":"ports",
>
>   "type":1,
>
>   "ranges":{
>
> "range":[
>
>   {
>
> "begin":31002,
>
> "end":31002
>
>   }
>
> ]
>
>   }
>
> }
>
>   ],
>
>   "command":{
>
> "container":{
>
>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>
> },
>
> "shell":false
>
>   },
>
>   "container":{
>
> "type":1,
>
> "docker":{
>
>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>
>   "network":2,
>
>   "port_mappings":[
>
> {
>
>   "host_port":31002,
>
>   "container_port":8000,
>
>   "protocol":"tcp"
>
> }
>
>   ],
>
>   "privileged":false,
>
>   "parameters":[],
>
>   "force_pull_image":false
>
> }
>
>   },
>
>   "health_check":{
>
> "delay_seconds":5,
>
> "interval_seconds":10,
>
> "timeout_seconds":10,
>
> "consecutive_failures":3,
>
> "grace_period_seconds":0,
>
> "command":{
>
>   "shell":true,
>
>   "value":"sleep 5",
>
>   "user":"root"
>
> }
>
>   }
>
> }
>
>
I have searched all machines and containers to see if they ever run the
command (in this case `sleep 5`), but have not found any indication that it
is being executed.

In the mesos src code the health-checks are invoked from
src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does this
mean that health-checks are only supported for custom executors and not for
docker tasks?

What I am trying to accomplish is to have the 0/non-zero exit-status of a
health-check command translate to task health.

Thanks!
Jay

Re: Can health-checks be run by Mesos for docker tasks?

[Timothy Chen]

Hi Jay, 

We just added health check support for docker tasks that's in master but not 
yet released. It will run docker exec with the command you provided as health 
checks.

It should be in the next release.

Thanks!

Tim


> On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
> 
> Does Mesos support health checks for docker image tasks?  Mesos seems to be 
> ignoring the TaskInfo.HealthCheck field for me.
> 
> Example TaskInfo JSON received back from Mesos:
> 
>>> {
>>>   "name":"hello-app.web.v3",
>>>   "task_id":{
>>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>>>   },
>>>   "slave_id":{
>>> "value":"20150924-210922-1608624320-5050-1792-S1"
>>>   },
>>>   "resources":[
>>> {
>>>   "name":"cpus",
>>>   "type":0,
>>>   "scalar":{
>>> "value":0.1
>>>   }
>>> },
>>> {
>>>   "name":"mem",
>>>   "type":0,
>>>   "scalar":{
>>> "value":256
>>>   }
>>> },
>>> {
>>>   "name":"ports",
>>>   "type":1,
>>>   "ranges":{
>>> "range":[
>>>   {
>>> "begin":31002,
>>> "end":31002
>>>   }
>>> ]
>>>   }
>>> }
>>>   ],
>>>   "command":{
>>> "container":{
>>>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>>> },
>>> "shell":false
>>>   },
>>>   "container":{
>>> "type":1,
>>> "docker":{
>>>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>>>   "network":2,
>>>   "port_mappings":[
>>> {
>>>   "host_port":31002,
>>>   "container_port":8000,
>>>   "protocol":"tcp"
>>> }
>>>   ],
>>>   "privileged":false,
>>>   "parameters":[],
>>>   "force_pull_image":false
>>> }
>>>   },
>>>   "health_check":{
>>> "delay_seconds":5,
>>> "interval_seconds":10,
>>> "timeout_seconds":10,
>>> "consecutive_failures":3,
>>> "grace_period_seconds":0,
>>> "command":{
>>>   "shell":true,
>>>   "value":"sleep 5",
>>>   "user":"root"
>>> }
>>>   }
>>> }
> 
> I have searched all machines and containers to see if they ever run the 
> command (in this case `sleep 5`), but have not found any indication that it 
> is being executed.
> 
> In the mesos src code the health-checks are invoked from 
> src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does this mean 
> that health-checks are only supported for custom executors and not for docker 
> tasks?
> 
> What I am trying to accomplish is to have the 0/non-zero exit-status of a 
> health-check command translate to task health.
> 
> Thanks!
> Jay

Re: Can health-checks be run by Mesos for docker tasks?

[Jay Taylor]

Great, thanks for the quick reply Tim!

Do you know if there is a branch I can checkout to test it out?

On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen  wrote:

> Hi Jay,
>
> We just added health check support for docker tasks that's in master but
> not yet released. It will run docker exec with the command you provided as
> health checks.
>
> It should be in the next release.
>
> Thanks!
>
> Tim
>
>
> On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
>
> Does Mesos support health checks for docker image tasks?  Mesos seems to
> be ignoring the TaskInfo.HealthCheck field for me.
>
> Example TaskInfo JSON received back from Mesos:
>
> {
>>
>>   "name":"hello-app.web.v3",
>>
>>   "task_id":{
>>
>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>>
>>   },
>>
>>   "slave_id":{
>>
>> "value":"20150924-210922-1608624320-5050-1792-S1"
>>
>>   },
>>
>>   "resources":[
>>
>> {
>>
>>   "name":"cpus",
>>
>>   "type":0,
>>
>>   "scalar":{
>>
>> "value":0.1
>>
>>   }
>>
>> },
>>
>> {
>>
>>   "name":"mem",
>>
>>   "type":0,
>>
>>   "scalar":{
>>
>> "value":256
>>
>>   }
>>
>> },
>>
>> {
>>
>>   "name":"ports",
>>
>>   "type":1,
>>
>>   "ranges":{
>>
>> "range":[
>>
>>   {
>>
>> "begin":31002,
>>
>> "end":31002
>>
>>   }
>>
>> ]
>>
>>   }
>>
>> }
>>
>>   ],
>>
>>   "command":{
>>
>> "container":{
>>
>>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>>
>> },
>>
>> "shell":false
>>
>>   },
>>
>>   "container":{
>>
>> "type":1,
>>
>> "docker":{
>>
>>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>>
>>   "network":2,
>>
>>   "port_mappings":[
>>
>> {
>>
>>   "host_port":31002,
>>
>>   "container_port":8000,
>>
>>   "protocol":"tcp"
>>
>> }
>>
>>   ],
>>
>>   "privileged":false,
>>
>>   "parameters":[],
>>
>>   "force_pull_image":false
>>
>> }
>>
>>   },
>>
>>   "health_check":{
>>
>> "delay_seconds":5,
>>
>> "interval_seconds":10,
>>
>> "timeout_seconds":10,
>>
>> "consecutive_failures":3,
>>
>> "grace_period_seconds":0,
>>
>> "command":{
>>
>>   "shell":true,
>>
>>   "value":"sleep 5",
>>
>>   "user":"root"
>>
>> }
>>
>>   }
>>
>> }
>>
>>
> I have searched all machines and containers to see if they ever run the
> command (in this case `sleep 5`), but have not found any indication that it
> is being executed.
>
> In the mesos src code the health-checks are invoked from
> src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does this
> mean that health-checks are only supported for custom executors and not for
> docker tasks?
>
> What I am trying to accomplish is to have the 0/non-zero exit-status of a
> health-check command translate to task health.
>
> Thanks!
> Jay
>
>

Re: Can health-checks be run by Mesos for docker tasks?

[Jay Taylor]

Oops- Now I see you already said it's in master.  I'll look there :)

Thanks again!

On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:

> Great, thanks for the quick reply Tim!
>
> Do you know if there is a branch I can checkout to test it out?
>
> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen  wrote:
>
>> Hi Jay,
>>
>> We just added health check support for docker tasks that's in master but
>> not yet released. It will run docker exec with the command you provided as
>> health checks.
>>
>> It should be in the next release.
>>
>> Thanks!
>>
>> Tim
>>
>>
>> On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
>>
>> Does Mesos support health checks for docker image tasks?  Mesos seems to
>> be ignoring the TaskInfo.HealthCheck field for me.
>>
>> Example TaskInfo JSON received back from Mesos:
>>
>> {
>>>
>>>   "name":"hello-app.web.v3",
>>>
>>>   "task_id":{
>>>
>>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>>>
>>>   },
>>>
>>>   "slave_id":{
>>>
>>> "value":"20150924-210922-1608624320-5050-1792-S1"
>>>
>>>   },
>>>
>>>   "resources":[
>>>
>>> {
>>>
>>>   "name":"cpus",
>>>
>>>   "type":0,
>>>
>>>   "scalar":{
>>>
>>> "value":0.1
>>>
>>>   }
>>>
>>> },
>>>
>>> {
>>>
>>>   "name":"mem",
>>>
>>>   "type":0,
>>>
>>>   "scalar":{
>>>
>>> "value":256
>>>
>>>   }
>>>
>>> },
>>>
>>> {
>>>
>>>   "name":"ports",
>>>
>>>   "type":1,
>>>
>>>   "ranges":{
>>>
>>> "range":[
>>>
>>>   {
>>>
>>> "begin":31002,
>>>
>>> "end":31002
>>>
>>>   }
>>>
>>> ]
>>>
>>>   }
>>>
>>> }
>>>
>>>   ],
>>>
>>>   "command":{
>>>
>>> "container":{
>>>
>>>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>>>
>>> },
>>>
>>> "shell":false
>>>
>>>   },
>>>
>>>   "container":{
>>>
>>> "type":1,
>>>
>>> "docker":{
>>>
>>>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>>>
>>>   "network":2,
>>>
>>>   "port_mappings":[
>>>
>>> {
>>>
>>>   "host_port":31002,
>>>
>>>   "container_port":8000,
>>>
>>>   "protocol":"tcp"
>>>
>>> }
>>>
>>>   ],
>>>
>>>   "privileged":false,
>>>
>>>   "parameters":[],
>>>
>>>   "force_pull_image":false
>>>
>>> }
>>>
>>>   },
>>>
>>>   "health_check":{
>>>
>>> "delay_seconds":5,
>>>
>>> "interval_seconds":10,
>>>
>>> "timeout_seconds":10,
>>>
>>> "consecutive_failures":3,
>>>
>>> "grace_period_seconds":0,
>>>
>>> "command":{
>>>
>>>   "shell":true,
>>>
>>>   "value":"sleep 5",
>>>
>>>   "user":"root"
>>>
>>> }
>>>
>>>   }
>>>
>>> }
>>>
>>>
>> I have searched all machines and containers to see if they ever run the
>> command (in this case `sleep 5`), but have not found any indication that it
>> is being executed.
>>
>> In the mesos src code the health-checks are invoked from
>> src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does this
>> mean that health-checks are only supported for custom executors and not for
>> docker tasks?
>>
>> What I am trying to accomplish is to have the 0/non-zero exit-status of a
>> health-check command translate to task health.
>>
>> Thanks!
>> Jay
>>
>>
>

Re: Can health-checks be run by Mesos for docker tasks?

[haosdent]

I remember 0.23.1 and 0.24.1 contains this backport, let me double check.

On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor  wrote:

> Oops- Now I see you already said it's in master.  I'll look there :)
>
> Thanks again!
>
> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:
>
>> Great, thanks for the quick reply Tim!
>>
>> Do you know if there is a branch I can checkout to test it out?
>>
>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen  wrote:
>>
>>> Hi Jay,
>>>
>>> We just added health check support for docker tasks that's in master but
>>> not yet released. It will run docker exec with the command you provided as
>>> health checks.
>>>
>>> It should be in the next release.
>>>
>>> Thanks!
>>>
>>> Tim
>>>
>>>
>>> On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
>>>
>>> Does Mesos support health checks for docker image tasks?  Mesos seems to
>>> be ignoring the TaskInfo.HealthCheck field for me.
>>>
>>> Example TaskInfo JSON received back from Mesos:
>>>
>>> {

   "name":"hello-app.web.v3",

   "task_id":{

 "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"

   },

   "slave_id":{

 "value":"20150924-210922-1608624320-5050-1792-S1"

   },

   "resources":[

 {

   "name":"cpus",

   "type":0,

   "scalar":{

 "value":0.1

   }

 },

 {

   "name":"mem",

   "type":0,

   "scalar":{

 "value":256

   }

 },

 {

   "name":"ports",

   "type":1,

   "ranges":{

 "range":[

   {

 "begin":31002,

 "end":31002

   }

 ]

   }

 }

   ],

   "command":{

 "container":{

   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"

 },

 "shell":false

   },

   "container":{

 "type":1,

 "docker":{

   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",

   "network":2,

   "port_mappings":[

 {

   "host_port":31002,

   "container_port":8000,

   "protocol":"tcp"

 }

   ],

   "privileged":false,

   "parameters":[],

   "force_pull_image":false

 }

   },

   "health_check":{

 "delay_seconds":5,

 "interval_seconds":10,

 "timeout_seconds":10,

 "consecutive_failures":3,

 "grace_period_seconds":0,

 "command":{

   "shell":true,

   "value":"sleep 5",

   "user":"root"

 }

   }

 }


>>> I have searched all machines and containers to see if they ever run the
>>> command (in this case `sleep 5`), but have not found any indication that it
>>> is being executed.
>>>
>>> In the mesos src code the health-checks are invoked from
>>> src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does this
>>> mean that health-checks are only supported for custom executors and not for
>>> docker tasks?
>>>
>>> What I am trying to accomplish is to have the 0/non-zero exit-status of
>>> a health-check command translate to task health.
>>>
>>> Thanks!
>>> Jay
>>>
>>>
>>
>


-- 
Best Regards,
Haosdent Huang

Re: Can health-checks be run by Mesos for docker tasks?

[haosdent]

yes, adam also help commit it to 0.23.1 and 0.24.1
https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0

https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7
Are you use one of this version?

On Wed, Oct 7, 2015 at 10:26 AM, haosdent  wrote:

> I remember 0.23.1 and 0.24.1 contains this backport, let me double check.
>
> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor  wrote:
>
>> Oops- Now I see you already said it's in master.  I'll look there :)
>>
>> Thanks again!
>>
>> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:
>>
>>> Great, thanks for the quick reply Tim!
>>>
>>> Do you know if there is a branch I can checkout to test it out?
>>>
>>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen  wrote:
>>>
 Hi Jay,

 We just added health check support for docker tasks that's in master
 but not yet released. It will run docker exec with the command you provided
 as health checks.

 It should be in the next release.

 Thanks!

 Tim


 On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:

 Does Mesos support health checks for docker image tasks?  Mesos seems
 to be ignoring the TaskInfo.HealthCheck field for me.

 Example TaskInfo JSON received back from Mesos:

 {
>
>   "name":"hello-app.web.v3",
>
>   "task_id":{
>
> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>
>   },
>
>   "slave_id":{
>
> "value":"20150924-210922-1608624320-5050-1792-S1"
>
>   },
>
>   "resources":[
>
> {
>
>   "name":"cpus",
>
>   "type":0,
>
>   "scalar":{
>
> "value":0.1
>
>   }
>
> },
>
> {
>
>   "name":"mem",
>
>   "type":0,
>
>   "scalar":{
>
> "value":256
>
>   }
>
> },
>
> {
>
>   "name":"ports",
>
>   "type":1,
>
>   "ranges":{
>
> "range":[
>
>   {
>
> "begin":31002,
>
> "end":31002
>
>   }
>
> ]
>
>   }
>
> }
>
>   ],
>
>   "command":{
>
> "container":{
>
>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>
> },
>
> "shell":false
>
>   },
>
>   "container":{
>
> "type":1,
>
> "docker":{
>
>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>
>   "network":2,
>
>   "port_mappings":[
>
> {
>
>   "host_port":31002,
>
>   "container_port":8000,
>
>   "protocol":"tcp"
>
> }
>
>   ],
>
>   "privileged":false,
>
>   "parameters":[],
>
>   "force_pull_image":false
>
> }
>
>   },
>
>   "health_check":{
>
> "delay_seconds":5,
>
> "interval_seconds":10,
>
> "timeout_seconds":10,
>
> "consecutive_failures":3,
>
> "grace_period_seconds":0,
>
> "command":{
>
>   "shell":true,
>
>   "value":"sleep 5",
>
>   "user":"root"
>
> }
>
>   }
>
> }
>
>
 I have searched all machines and containers to see if they ever run the
 command (in this case `sleep 5`), but have not found any indication that it
 is being executed.

 In the mesos src code the health-checks are invoked from
 src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does this
 mean that health-checks are only supported for custom executors and not for
 docker tasks?

 What I am trying to accomplish is to have the 0/non-zero exit-status of
 a health-check command translate to task health.

 Thanks!
 Jay


>>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 
Best Regards,
Haosdent Huang

Re: Can health-checks be run by Mesos for docker tasks?

[haosdent]

Could you use "exit 1" instead of "sleep 5" to see whether could see
unhealthy status in your task stdout/stderr.

On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor  wrote:

> My current version is 0.24.1.
>
> On Tue, Oct 6, 2015 at 7:30 PM, haosdent  wrote:
>
>> yes, adam also help commit it to 0.23.1 and 0.24.1
>> https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0
>>
>> https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7
>> Are you use one of this version?
>>
>> On Wed, Oct 7, 2015 at 10:26 AM, haosdent  wrote:
>>
>>> I remember 0.23.1 and 0.24.1 contains this backport, let me double check.
>>>
>>> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor  wrote:
>>>
 Oops- Now I see you already said it's in master.  I'll look there :)

 Thanks again!

 On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor  wrote:

> Great, thanks for the quick reply Tim!
>
> Do you know if there is a branch I can checkout to test it out?
>
> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen 
> wrote:
>
>> Hi Jay,
>>
>> We just added health check support for docker tasks that's in master
>> but not yet released. It will run docker exec with the command you 
>> provided
>> as health checks.
>>
>> It should be in the next release.
>>
>> Thanks!
>>
>> Tim
>>
>>
>> On Oct 6, 2015, at 6:49 PM, Jay Taylor  wrote:
>>
>> Does Mesos support health checks for docker image tasks?  Mesos seems
>> to be ignoring the TaskInfo.HealthCheck field for me.
>>
>> Example TaskInfo JSON received back from Mesos:
>>
>> {
>>>
>>>   "name":"hello-app.web.v3",
>>>
>>>   "task_id":{
>>>
>>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec"
>>>
>>>   },
>>>
>>>   "slave_id":{
>>>
>>> "value":"20150924-210922-1608624320-5050-1792-S1"
>>>
>>>   },
>>>
>>>   "resources":[
>>>
>>> {
>>>
>>>   "name":"cpus",
>>>
>>>   "type":0,
>>>
>>>   "scalar":{
>>>
>>> "value":0.1
>>>
>>>   }
>>>
>>> },
>>>
>>> {
>>>
>>>   "name":"mem",
>>>
>>>   "type":0,
>>>
>>>   "scalar":{
>>>
>>> "value":256
>>>
>>>   }
>>>
>>> },
>>>
>>> {
>>>
>>>   "name":"ports",
>>>
>>>   "type":1,
>>>
>>>   "ranges":{
>>>
>>> "range":[
>>>
>>>   {
>>>
>>> "begin":31002,
>>>
>>> "end":31002
>>>
>>>   }
>>>
>>> ]
>>>
>>>   }
>>>
>>> }
>>>
>>>   ],
>>>
>>>   "command":{
>>>
>>> "container":{
>>>
>>>   "image":"docker-services1a:5000/test/app-81-1-hello-app-103"
>>>
>>> },
>>>
>>> "shell":false
>>>
>>>   },
>>>
>>>   "container":{
>>>
>>> "type":1,
>>>
>>> "docker":{
>>>
>>>   "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103",
>>>
>>>   "network":2,
>>>
>>>   "port_mappings":[
>>>
>>> {
>>>
>>>   "host_port":31002,
>>>
>>>   "container_port":8000,
>>>
>>>   "protocol":"tcp"
>>>
>>> }
>>>
>>>   ],
>>>
>>>   "privileged":false,
>>>
>>>   "parameters":[],
>>>
>>>   "force_pull_image":false
>>>
>>> }
>>>
>>>   },
>>>
>>>   "health_check":{
>>>
>>> "delay_seconds":5,
>>>
>>> "interval_seconds":10,
>>>
>>> "timeout_seconds":10,
>>>
>>> "consecutive_failures":3,
>>>
>>> "grace_period_seconds":0,
>>>
>>> "command":{
>>>
>>>   "shell":true,
>>>
>>>   "value":"sleep 5",
>>>
>>>   "user":"root"
>>>
>>> }
>>>
>>>   }
>>>
>>> }
>>>
>>>
>> I have searched all machines and containers to see if they ever run
>> the command (in this case `sleep 5`), but have not found any indication
>> that it is being executed.
>>
>> In the mesos src code the health-checks are invoked from
>> src/launcher/executor.cpp CommandExecutorProcess::launchTask.  Does this
>> mean that health-checks are only supported for custom executors and not 
>> for
>> docker tasks?
>>
>> What I am trying to accomplish is to have the 0/non-zero exit-status
>> of a health-check command translate to task health.
>>
>> Thanks!
>> Jay
>>
>>
>

>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>

Old docker version deployed

[Paul Wolfe]

Hello all,



I'm new to this list, so please let me know if there is a better/more 
appropriate forum for this question.



We are currently experimenting with marathon and mesos for deploying a simple 
webapp.  We ship the app as a docker container.



Sporadically (ie 1 out of 100) we find an old version of the app is deployed.  
It is obvious from the logs and the appearance of the GUI that the version is 
old.  If I download and run the docker container locally, I see it is indeed 
the latest version of the code.  That leads me to believe that somewhere in the 
marathon deploy or the mesos running of the image, versions are getting 
confused.



I guess my first question is what additional information can I get from 
marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but 
haven't been able to garner anything interesting there.



Thanks for any help!

Paul Wolfe




The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.

RE: Old docker version deployed

[Paul Wolfe]

No different tags.

From: Rad Gruchalski [mailto:ra...@gruchalski.com]
Sent: Tuesday, October 06, 2015 11:39 AM
To: user@mesos.apache.org
Subject: Re: Old docker version deployed

Paul,

Are you using the same tag every time?

Kind regards,

Radek Gruchalski

ra...@gruchalski.com

de.linkedin.com/in/radgruchalski/

Confidentiality:
This communication is intended for the above-named person and may be 
confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must 
you copy or show it to anyone; please delete/destroy and inform the sender 
immediately.

On Tuesday, 6 October 2015 at 11:37, haosdent wrote:
You could see the stdout/stderr of your container from mesos webui.

On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe 
> wrote:


Hello all,



I'm new to this list, so please let me know if there is a better/more 
appropriate forum for this question.



We are currently experimenting with marathon and mesos for deploying a simple 
webapp.  We ship the app as a docker container.



Sporadically (ie 1 out of 100) we find an old version of the app is deployed.  
It is obvious from the logs and the appearance of the GUI that the version is 
old.  If I download and run the docker container locally, I see it is indeed 
the latest version of the code.  That leads me to believe that somewhere in the 
marathon deploy or the mesos running of the image, versions are getting 
confused.



I guess my first question is what additional information can I get from 
marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but 
haven't been able to garner anything interesting there.



Thanks for any help!

Paul Wolfe





The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.



--
Best Regards,
Haosdent Huang




The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.

RE: Old docker version deployed

[Paul Wolfe]

I do see the stdout in the webgui, which is how I can confirm the old version 
is deployed.

What I need is some information about what version/tag of the image mesos is 
using.

From: haosdent [mailto:haosd...@gmail.com]
Sent: Tuesday, October 06, 2015 11:37 AM
To: user@mesos.apache.org
Subject: Re: Old docker version deployed

You could see the stdout/stderr of your container from mesos webui.

On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe 
> wrote:

Hello all,



I'm new to this list, so please let me know if there is a better/more 
appropriate forum for this question.



We are currently experimenting with marathon and mesos for deploying a simple 
webapp.  We ship the app as a docker container.



Sporadically (ie 1 out of 100) we find an old version of the app is deployed.  
It is obvious from the logs and the appearance of the GUI that the version is 
old.  If I download and run the docker container locally, I see it is indeed 
the latest version of the code.  That leads me to believe that somewhere in the 
marathon deploy or the mesos running of the image, versions are getting 
confused.



I guess my first question is what additional information can I get from 
marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but 
haven't been able to garner anything interesting there.



Thanks for any help!

Paul Wolfe




The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.



--
Best Regards,
Haosdent Huang



The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.

RE: Old docker version deployed

[Paul Wolfe]

My marathon deploy json:

{
 "type": "DOCKER",
  "volumes": [
{
  "containerPath": "/home/myapp /log",
  "hostPath": "/home",
  "mode": "RW"
}
  ],
  "docker": {
"image": "docker-registry:8080/myapp:86",
"network": "BRIDGE",
"portMappings": [
  {
"containerPort": 80,
"hostPort": 0,
"servicePort": 80,
"protocol": "tcp"
  }
],
"privileged": false,
"parameters": [],
"forcePullImage": false
  }
}


From: Paul Wolfe [mailto:paul.wo...@imc.nl]
Sent: Tuesday, October 06, 2015 11:39 AM
To: user@mesos.apache.org
Subject: RE: Old docker version deployed

No different tags.

From: Rad Gruchalski [mailto:ra...@gruchalski.com]
Sent: Tuesday, October 06, 2015 11:39 AM
To: user@mesos.apache.org
Subject: Re: Old docker version deployed

Paul,

Are you using the same tag every time?

Kind regards,

Radek Gruchalski

ra...@gruchalski.com

de.linkedin.com/in/radgruchalski/

Confidentiality:
This communication is intended for the above-named person and may be 
confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must 
you copy or show it to anyone; please delete/destroy and inform the sender 
immediately.

On Tuesday, 6 October 2015 at 11:37, haosdent wrote:
You could see the stdout/stderr of your container from mesos webui.

On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe 
> wrote:

Hello all,



I'm new to this list, so please let me know if there is a better/more 
appropriate forum for this question.



We are currently experimenting with marathon and mesos for deploying a simple 
webapp.  We ship the app as a docker container.



Sporadically (ie 1 out of 100) we find an old version of the app is deployed.  
It is obvious from the logs and the appearance of the GUI that the version is 
old.  If I download and run the docker container locally, I see it is indeed 
the latest version of the code.  That leads me to believe that somewhere in the 
marathon deploy or the mesos running of the image, versions are getting 
confused.



I guess my first question is what additional information can I get from 
marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but 
haven't been able to garner anything interesting there.



Thanks for any help!

Paul Wolfe





The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.



--
Best Regards,
Haosdent Huang




The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.



The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / 

Re: Old docker version deployed

[haosdent]

You could see the stdout/stderr of your container from mesos webui.

On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe  wrote:

> Hello all,
>
>
>
> I'm new to this list, so please let me know if there is a better/more
> appropriate forum for this question.
>
>
>
> We are currently experimenting with marathon and mesos for deploying a
> simple webapp.  We ship the app as a docker container.
>
>
>
> Sporadically (ie 1 out of 100) we find an old version of the app is
> deployed.  It is obvious from the logs and the appearance of the GUI that
> the version is old.  If I download and run the docker container locally, I
> see it is indeed the latest version of the code.  That leads me to believe
> that somewhere in the marathon deploy or the mesos running of the image,
> versions are getting confused.
>
>
>
> I guess my first question is what additional information can I get from
> marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but
> haven't been able to garner anything interesting there.
>
>
>
> Thanks for any help!
>
> Paul Wolfe
>
>
>
> --
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. In the event of incomplete or incorrect
> transmission, please return the e-mail to the sender and permanently delete
> this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan
> attachments before opening them.
>



-- 
Best Regards,
Haosdent Huang

Re: Old docker version deployed

[Rad Gruchalski]

Paul,

Are you using the same tag every time?










Kind regards,

Radek Gruchalski

ra...@gruchalski.com (mailto:ra...@gruchalski.com)
 
(mailto:ra...@gruchalski.com)
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)

Confidentiality:
This communication is intended for the above-named person and may be 
confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must 
you copy or show it to anyone; please delete/destroy and inform the sender 
immediately.



On Tuesday, 6 October 2015 at 11:37, haosdent wrote:

> You could see the stdout/stderr of your container from mesos webui.
>  
> On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe  (mailto:paul.wo...@imc.nl)> wrote:
> > Hello all,  
> >   
> > I'm new to this list, so please let me know if there is a better/more 
> > appropriate forum for this question.
> >   
> > We are currently experimenting with marathon and mesos for deploying a 
> > simple webapp.  We ship the app as a docker container.
> >   
> > Sporadically (ie 1 out of 100) we find an old version of the app is 
> > deployed.  It is obvious from the logs and the appearance of the GUI that 
> > the version is old.  If I download and run the docker container locally, I 
> > see it is indeed the latest version of the code.  That leads me to believe 
> > that somewhere in the marathon deploy or the mesos running of the image, 
> > versions are getting confused.
> >   
> > I guess my first question is what additional information can I get from 
> > marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but 
> > haven't been able to garner anything interesting there.
> >   
> > Thanks for any help!
> > Paul Wolfe
> >   
> >  
> >  
> >  
> > The information in this e-mail is intended only for the person or entity to 
> > which it is addressed.
> >  
> > It may contain confidential and /or privileged material. If someone other 
> > than the intended recipient should receive this e-mail, he / she shall not 
> > be entitled to read, disseminate, disclose or duplicate it.
> >  
> > If you receive this e-mail unintentionally, please inform us immediately by 
> > "reply" and then delete it from your system. Although this information has 
> > been compiled with great care, neither IMC Financial Markets & Asset 
> > Management nor any of its related entities shall accept any responsibility 
> > for any errors, omissions or other inaccuracies in this information or for 
> > the consequences thereof, nor shall it be bound in any way by the contents 
> > of this e-mail or its attachments. In the event of incomplete or incorrect 
> > transmission, please return the e-mail to the sender and permanently delete 
> > this message and any attachments.
> >  
> > Messages and attachments are scanned for all known viruses. Always scan 
> > attachments before opening them.
>  
>  
>  
> --  
> Best Regards,
> Haosdent Huang  

Re: Old docker version deployed

[haosdent]

I don't think mesos log "version/tag of the image". When mesos start a
docker container, always use your image name
"docker-registry:8080/myapp:86" as pull and run parameters. I think maybe
some machines have problems to connect your image registry.

On Tue, Oct 6, 2015 at 5:40 PM, Paul Wolfe  wrote:

> My marathon deploy json:
>
>
>
> {
>
>  "type": "DOCKER",
>
>   "volumes": [
>
> {
>
>   "containerPath": "/home/myapp /log",
>
>   "hostPath": "/home",
>
>   "mode": "RW"
>
> }
>
>   ],
>
>   "docker": {
>
> "image": "docker-registry:8080/myapp:86",
>
> "network": "BRIDGE",
>
> "portMappings": [
>
>   {
>
> "containerPort": 80,
>
> "hostPort": 0,
>
> "servicePort": 80,
>
> "protocol": "tcp"
>
>   }
>
> ],
>
> "privileged": false,
>
> "parameters": [],
>
> "forcePullImage": false
>
>   }
>
> }
>
>
>
>
>
> *From:* Paul Wolfe [mailto:paul.wo...@imc.nl]
> *Sent:* Tuesday, October 06, 2015 11:39 AM
> *To:* user@mesos.apache.org
> *Subject:* RE: Old docker version deployed
>
>
>
> No different tags.
>
>
>
> *From:* Rad Gruchalski [mailto:ra...@gruchalski.com ]
>
> *Sent:* Tuesday, October 06, 2015 11:39 AM
> *To:* user@mesos.apache.org
> *Subject:* Re: Old docker version deployed
>
>
>
> Paul,
>
>
>
> Are you using the same tag every time?
>
> Kind regards,
> Radek Gruchalski
> ra...@gruchalski.com 
> de.linkedin.com/in/radgruchalski/
>
>
> *Confidentiality: *This communication is intended for the above-named
> person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor
> must you copy or show it to anyone; please delete/destroy and inform the
> sender immediately.
>
> On Tuesday, 6 October 2015 at 11:37, haosdent wrote:
>
> You could see the stdout/stderr of your container from mesos webui.
>
>
>
> On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe  wrote:
>
> Hello all,
>
>
>
> I'm new to this list, so please let me know if there is a better/more
> appropriate forum for this question.
>
>
>
> We are currently experimenting with marathon and mesos for deploying a
> simple webapp.  We ship the app as a docker container.
>
>
>
> Sporadically (ie 1 out of 100) we find an old version of the app is
> deployed.  It is obvious from the logs and the appearance of the GUI that
> the version is old.  If I download and run the docker container locally, I
> see it is indeed the latest version of the code.  That leads me to believe
> that somewhere in the marathon deploy or the mesos running of the image,
> versions are getting confused.
>
>
>
> I guess my first question is what additional information can I get from
> marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but
> haven't been able to garner anything interesting there.
>
>
>
> Thanks for any help!
>
> Paul Wolfe
>
>
>
>
> --
>
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. In the event of incomplete or incorrect
> transmission, please return the e-mail to the sender and permanently delete
> this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan
> attachments before opening them.
>
>
>
>
>
> --
>
> Best Regards,
>
> Haosdent Huang
>
>
>
>
> --
>
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. 

RE: Old docker version deployed

[Paul Wolfe]

Fair enough, although if that was the case would expect it to fail hard, not 
randomly run an old image.

One thing I did notice was that on the master box, docker images misses the 
version that should have been deployed (ie has version 77 and 79, but no 78)

From: haosdent [mailto:haosd...@gmail.com]
Sent: Tuesday, October 06, 2015 11:52 AM
To: user@mesos.apache.org
Subject: Re: Old docker version deployed

I don't think mesos log "version/tag of the image". When mesos start a docker 
container, always use your image name "docker-registry:8080/myapp:86" as pull 
and run parameters. I think maybe some machines have problems to connect your 
image registry.

On Tue, Oct 6, 2015 at 5:40 PM, Paul Wolfe 
> wrote:
My marathon deploy json:

{
 "type": "DOCKER",
  "volumes": [
{
  "containerPath": "/home/myapp /log",
  "hostPath": "/home",
  "mode": "RW"
}
  ],
  "docker": {
"image": "docker-registry:8080/myapp:86",
"network": "BRIDGE",
"portMappings": [
  {
"containerPort": 80,
"hostPort": 0,
"servicePort": 80,
"protocol": "tcp"
  }
],
"privileged": false,
"parameters": [],
"forcePullImage": false
  }
}


From: Paul Wolfe [mailto:paul.wo...@imc.nl]
Sent: Tuesday, October 06, 2015 11:39 AM
To: user@mesos.apache.org
Subject: RE: Old docker version deployed

No different tags.

From: Rad Gruchalski [mailto:ra...@gruchalski.com]
Sent: Tuesday, October 06, 2015 11:39 AM
To: user@mesos.apache.org
Subject: Re: Old docker version deployed

Paul,

Are you using the same tag every time?

Kind regards,

Radek Gruchalski

ra...@gruchalski.com

de.linkedin.com/in/radgruchalski/

Confidentiality:
This communication is intended for the above-named person and may be 
confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must 
you copy or show it to anyone; please delete/destroy and inform the sender 
immediately.

On Tuesday, 6 October 2015 at 11:37, haosdent wrote:
You could see the stdout/stderr of your container from mesos webui.

On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe 
> wrote:

Hello all,



I'm new to this list, so please let me know if there is a better/more 
appropriate forum for this question.



We are currently experimenting with marathon and mesos for deploying a simple 
webapp.  We ship the app as a docker container.



Sporadically (ie 1 out of 100) we find an old version of the app is deployed.  
It is obvious from the logs and the appearance of the GUI that the version is 
old.  If I download and run the docker container locally, I see it is indeed 
the latest version of the code.  That leads me to believe that somewhere in the 
marathon deploy or the mesos running of the image, versions are getting 
confused.



I guess my first question is what additional information can I get from 
marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but 
haven't been able to garner anything interesting there.



Thanks for any help!

Paul Wolfe





The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this message and any 
attachments.

Messages and attachments are scanned for all known viruses. Always scan 
attachments before opening them.



--
Best Regards,
Haosdent Huang




The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC 

Re: Old docker version deployed

[haosdent]

Randomly run an old image looks strange, is it because your image registry
have problems in some cases?

On Tue, Oct 6, 2015 at 5:53 PM, Paul Wolfe  wrote:

> Fair enough, although if that was the case would expect it to fail hard,
> not randomly run an old image.
>
>
>
> One thing I did notice was that on the master box, docker images misses
> the version that should have been deployed (ie has version 77 and 79, but
> no 78)
>
>
>
> *From:* haosdent [mailto:haosd...@gmail.com]
> *Sent:* Tuesday, October 06, 2015 11:52 AM
>
> *To:* user@mesos.apache.org
> *Subject:* Re: Old docker version deployed
>
>
>
> I don't think mesos log "version/tag of the image". When mesos start a
> docker container, always use your image name
> "docker-registry:8080/myapp:86" as pull and run parameters. I think maybe
> some machines have problems to connect your image registry.
>
>
>
> On Tue, Oct 6, 2015 at 5:40 PM, Paul Wolfe  wrote:
>
> My marathon deploy json:
>
>
>
> {
>
>  "type": "DOCKER",
>
>   "volumes": [
>
> {
>
>   "containerPath": "/home/myapp /log",
>
>   "hostPath": "/home",
>
>   "mode": "RW"
>
> }
>
>   ],
>
>   "docker": {
>
> "image": "docker-registry:8080/myapp:86",
>
> "network": "BRIDGE",
>
> "portMappings": [
>
>   {
>
> "containerPort": 80,
>
> "hostPort": 0,
>
> "servicePort": 80,
>
> "protocol": "tcp"
>
>   }
>
> ],
>
> "privileged": false,
>
> "parameters": [],
>
> "forcePullImage": false
>
>   }
>
> }
>
>
>
>
>
> *From:* Paul Wolfe [mailto:paul.wo...@imc.nl]
> *Sent:* Tuesday, October 06, 2015 11:39 AM
> *To:* user@mesos.apache.org
> *Subject:* RE: Old docker version deployed
>
>
>
> No different tags.
>
>
>
> *From:* Rad Gruchalski [mailto:ra...@gruchalski.com ]
>
> *Sent:* Tuesday, October 06, 2015 11:39 AM
> *To:* user@mesos.apache.org
> *Subject:* Re: Old docker version deployed
>
>
>
> Paul,
>
>
>
> Are you using the same tag every time?
>
> Kind regards,
> Radek Gruchalski
> ra...@gruchalski.com 
> de.linkedin.com/in/radgruchalski/
>
>
> *Confidentiality: *This communication is intended for the above-named
> person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor
> must you copy or show it to anyone; please delete/destroy and inform the
> sender immediately.
>
> On Tuesday, 6 October 2015 at 11:37, haosdent wrote:
>
> You could see the stdout/stderr of your container from mesos webui.
>
>
>
> On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe  wrote:
>
> Hello all,
>
>
>
> I'm new to this list, so please let me know if there is a better/more
> appropriate forum for this question.
>
>
>
> We are currently experimenting with marathon and mesos for deploying a
> simple webapp.  We ship the app as a docker container.
>
>
>
> Sporadically (ie 1 out of 100) we find an old version of the app is
> deployed.  It is obvious from the logs and the appearance of the GUI that
> the version is old.  If I download and run the docker container locally, I
> see it is indeed the latest version of the code.  That leads me to believe
> that somewhere in the marathon deploy or the mesos running of the image,
> versions are getting confused.
>
>
>
> I guess my first question is what additional information can I get from
> marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but
> haven't been able to garner anything interesting there.
>
>
>
> Thanks for any help!
>
> Paul Wolfe
>
>
>
>
> --
>
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. In the event of incomplete or incorrect
> transmission, please return the e-mail to the sender and permanently delete
> this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan
> attachments before opening them.
>
>
>
>
>
> --
>
> Best Regards,
>
> Haosdent Huang
>
>
>
>
> --
>
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If 

Re: Scheduling tasks based on dependancy

[Pradeep Kiruvale]

Hi Sharma,

Is this how you collect the network info from the VMs?

First you get the resource offers from the Mesos and then you collect the
network bandwidth info and then you use that for assigning for your tasks?
Or
The mesos-slave collects the resource information? But I don't see any code
to that and also the existing mesos-slave does not collects these resource
information by itself.

Am I missing something here?

Regards,
Pradeep

On 5 October 2015 at 18:28, Sharma Podila  wrote:

> Pradeep,
>
> We recently open sourced Fenzo  (wiki
> ) to handle these scenarios. We
> add a custom attribute for network bandwidth for each agent's "mesos-slave"
> command line. And we have Fenzo assign resources to tasks based on CPU,
> memory, disk, ports, and network bandwidth requirements. With Fenzo you can
> define affinity, locality, and any other custom scheduling objectives using
> plugins. Some of the plugins are already built in. It is also easy to add
> additional plugins to cover other objectives you care about.
>
> "Dependencies" can mean multiple things. Do you mean dependencies on
> certain attributes of resources/agents? Dependencies on where other tasks
> are assigned? All of these are covered. However, if you mean workflow type
> of dependencies on completion of other tasks, then, there are no built in
> plugins. You could write one using Fenzo. It is also common for such
> workflow dependencies to be covered by an entity external to the scheduler.
> Both techniques can be made to work.
>
> Fenzo has the concept of hard Vs soft constraints. You could specify, for
> example, resource affinity and/or task locality as a soft constraint or a
> hard constraint. See the wiki docs link I provided above for details.
>
> Sharma
>
>
> On Mon, Oct 5, 2015 at 8:21 AM, Pradeep Kiruvale <
> pradeepkiruv...@gmail.com> wrote:
>
>> Hi All,
>>
>> Are there any frameworks that exists with the Mesos to schedule the
>> bigger apps?
>> I mean to say scheduling a app which has many services and will not fit
>> into one physical node.
>>
>> Is there any frame work that can be used to
>>  schedule tasks based on the underlying hardware constraints like Network
>> bandwidth ?
>>  Schedule the tasks based on their dependencies and proximity to each
>> other in a cluster or a rack?
>>
>> Thanks & Regards,
>> Pradeep
>>
>
>

RE: Old docker version deployed

[Paul Wolfe]

Turns out it was a “bug” in docker. We found that running by hand the same tag 
(78) would randomly run version 18. Wouldn’t pull, even though the images 
wasn’t in the cache.

Upgrading from docker 1.7.1 to 1.8.2 seems to solve it, dangerous problem 
though…

From: Rad Gruchalski [mailto:ra...@gruchalski.com]
Sent: Tuesday, October 06, 2015 11:54 AM
To: user@mesos.apache.org
Subject: Re: Old docker version deployed

But if the image version is changed, this would fail. Because the image is 
neither locally, neither the registry is available.

Kind regards,

Radek Gruchalski

ra...@gruchalski.com

de.linkedin.com/in/radgruchalski/

Confidentiality:
This communication is intended for the above-named person and may be 
confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must 
you copy or show it to anyone; please delete/destroy and inform the sender 
immediately.

On Tuesday, 6 October 2015 at 11:51, haosdent wrote:
I don't think mesos log "version/tag of the image". When mesos start a docker 
container, always use your image name "docker-registry:8080/myapp:86" as pull 
and run parameters. I think maybe some machines have problems to connect your 
image registry.

On Tue, Oct 6, 2015 at 5:40 PM, Paul Wolfe 
> wrote:


My marathon deploy json:



{

 "type": "DOCKER",

  "volumes": [

{

  "containerPath": "/home/myapp /log",

  "hostPath": "/home",

  "mode": "RW"

}

  ],

  "docker": {

"image": "docker-registry:8080/myapp:86",

"network": "BRIDGE",

"portMappings": [

  {

"containerPort": 80,

"hostPort": 0,

"servicePort": 80,

"protocol": "tcp"

  }

],

"privileged": false,

"parameters": [],

"forcePullImage": false

  }

}





From: Paul Wolfe [mailto:paul.wo...@imc.nl]
Sent: Tuesday, October 06, 2015 11:39 AM
To: user@mesos.apache.org
Subject: RE: Old docker version deployed



No different tags.



From: Rad Gruchalski [mailto:ra...@gruchalski.com]
Sent: Tuesday, October 06, 2015 11:39 AM
To: user@mesos.apache.org
Subject: Re: Old docker version deployed



Paul,



Are you using the same tag every time?

Kind regards,

Radek Gruchalski

ra...@gruchalski.com

de.linkedin.com/in/radgruchalski/

Confidentiality:
This communication is intended for the above-named person and may be 
confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must 
you copy or show it to anyone; please delete/destroy and inform the sender 
immediately.

On Tuesday, 6 October 2015 at 11:37, haosdent wrote:

You could see the stdout/stderr of your container from mesos webui.



On Tue, Oct 6, 2015 at 5:30 PM, Paul Wolfe 
> wrote:

Hello all,



I'm new to this list, so please let me know if there is a better/more 
appropriate forum for this question.



We are currently experimenting with marathon and mesos for deploying a simple 
webapp.  We ship the app as a docker container.



Sporadically (ie 1 out of 100) we find an old version of the app is deployed.  
It is obvious from the logs and the appearance of the GUI that the version is 
old.  If I download and run the docker container locally, I see it is indeed 
the latest version of the code.  That leads me to believe that somewhere in the 
marathon deploy or the mesos running of the image, versions are getting 
confused.



I guess my first question is what additional information can I get from 
marathon or mesos logs to help diagnose? I've checked the mesos-SLAVE.* but 
haven't been able to garner anything interesting there.



Thanks for any help!

Paul Wolfe







The information in this e-mail is intended only for the person or entity to 
which it is addressed.

It may contain confidential and /or privileged material. If someone other than 
the intended recipient should receive this e-mail, he / she shall not be 
entitled to read, disseminate, disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by 
"reply" and then delete it from your system. Although this information has been 
compiled with great care, neither IMC Financial Markets & Asset Management nor 
any of its related entities shall accept any responsibility for any errors, 
omissions or other inaccuracies in this information or for the consequences 
thereof, nor shall it be bound in any way by the contents of this e-mail or its 
attachments. In the event of incomplete or incorrect transmission, please 
return the e-mail to the sender and permanently delete this 

command executor stdout with status update

[Bijoy V]

Hello,
Is there a way we can add the default command executor output to the status
update lifecycle method of Scheduler. Can you please guide to achieve this
somehow
Thanks,
-Bijoy

Re: Scheduling tasks based on dependancy

[Pradeep Kiruvale]

Hi Guangya,

One doubt about the  --attributes=rackid:r1;groupid:g1 option.

How does the master provisions the resources? How will be the resource
offer?

Is it like (Rack 1 , G1, System)? how does this way of  doing resource
offer will help?

Can you please give me more information?


-Pradeep



On 5 October 2015 at 17:45, Guangya Liu  wrote:

> Hi Pradeep,
>
> I think that you can try Chronos and Marathon which can help you.
>
> *Marathon:* https://github.com/mesosphere/marathon
> You can try Marathon + Mesos + Mesos Resource Attribute
>
> When you start up mesos slave, uses --attributes option, here is an
> example:
> ./bin/mesos-slave.sh --master=9.21.61.21:5050 --quiet
> --log_dir=/tmp/mesos --attributes=rackid:r1;groupid:g1
> This basically defines two attributes for this mesos slave host. rackid
> with value r1 and groupid with value g1.
>
> marathon start -i "like_test" -C "sleep 100" -n 4 -c 1 -m 50 -o
> "rackid:LIKE:r1"
>
> this will place applications on the slave node whose rackid is r1
>
> *Chronos:* https://github.com/mesos/chronos , Chronos supports the
> definition of jobs triggered by the completion of other jobs. It supports
> arbitrarily long dependency chains.
>
> Thanks,
>
> Guangya
>
> On Mon, Oct 5, 2015 at 11:21 PM, Pradeep Kiruvale <
> pradeepkiruv...@gmail.com> wrote:
>
>> Hi All,
>>
>> Are there any frameworks that exists with the Mesos to schedule the
>> bigger apps?
>> I mean to say scheduling a app which has many services and will not fit
>> into one physical node.
>>
>> Is there any frame work that can be used to
>>  schedule tasks based on the underlying hardware constraints like Network
>> bandwidth ?
>>
>  Schedule the tasks based on their dependencies and proximity to each
>> other in a cluster or a rack?
>>
>> Thanks & Regards,
>> Pradeep
>>
>
>

Re: command executor stdout with status update

[haosdent]

I not suggest you add the stdout/stderr to status update. And you could get
stdout/stderr from mesos http endpoint.
https://github.com/apache/mesos/blob/master/src/cli/mesos-tail is a example
to show how get stdout/stderr through /files/read endpoint.

If you want to get status through you command output, I suggest you
strengthen your command to let it could return exit status according your
origin command output.

On Tue, Oct 6, 2015 at 6:55 PM, Bijoy V  wrote:

> Hello,
> Is there a way we can add the default command executor output to the
> status update lifecycle method of Scheduler. Can you please guide to
> achieve this somehow
> Thanks,
> -Bijoy
>



-- 
Best Regards,
Haosdent Huang

Broken images in documentation on web

[Brian Candler]

It seems several documentation pages now have broken images, for example:

http://mesos.apache.org/documentation/latest/external-containerizer/
(under " Container Lifecycle Sequence Diagrams")

http://mesos.apache.org/documentation/latest/oversubscription/
(under "How does it work?")

However, some pages are OK, e.g.
http://mesos.apache.org/documentation/latest/mesos-architecture/

Could someone investigate please?

Thanks,

Brian Candler.

Re: Securing executors

[Alexander Rojas]

Hi Paul,

I can refer you to the talk given by Adam Bordelon at MesosCon 
https://www.youtube.com/watch?v=G3sn1OLYDOE 
 

If you want to the short answer, the solution is to put a firewall around your 
cluster.

On a closer look on the port, it is the one used for message passing between 
the mesas-docker-executor and other mesos components.


> On 05 Oct 2015, at 19:04, Paul Bell  wrote:
> 
> Hi All,
> 
> I am running an nmap port scan on a Mesos agent node and noticed nmap 
> reporting an open TCP port at 50577.
> 
> Poking around some, I discovered exactly 5 mesos-docker-executor processes, 
> one for each of my 5 Docker containers, and each with an open listen port:
> 
> root 14131  3617  0 10:39 ?00:00:17 mesos-docker-executor 
> --container=mesos-20151002-172703-2450482247-5050-3014-S0.5563c65a-e33e-4287-8ce4-b2aa8116aa95
>  --docker=/usr/local/ecxmcc/weaveShim --help=false 
> --mapped_directory=/mnt/mesos/sandbox 
> --sandbox_directory=/tmp/mesos/slaves/20151002-172703-2450482247-5050-3014-S0/frameworks/20151002-172703-2450482247-5050-3014-/executors/postgres.ea2954fd-6b6e-11e5-8bef-56847afe9799/runs/5563c65a-e33e-4287-8ce4-b2aa8116aa95
>  --stop_timeout=15secs
> 
> I suppose that all of this is unsurprising. But I know of at least one big 
> customer who will without delay run Nmap or Nessus against my clustered 
> deployment.
> 
> So I am wondering what the best practices approach is to securing these open 
> ports. 
> 
> Thanks for your help.
> 
> -Paul
> 
> 
> 

Re: Securing executors

[Paul Bell]

Thanks, Alexander; I will check out the vid.

I kind of assumed that this port was used for exactly the purpose you
mention.

Is TLS a possibility here?

-Paul

On Tue, Oct 6, 2015 at 8:15 AM, Alexander Rojas 
wrote:

> Hi Paul,
>
> I can refer you to the talk given by Adam Bordelon at MesosCon
> https://www.youtube.com/watch?v=G3sn1OLYDOE
>
> If you want to the short answer, the solution is to put a firewall around
> your cluster.
>
> On a closer look on the port, it is the one used for message passing
> between the mesas-docker-executor and other mesos components.
>
>
> On 05 Oct 2015, at 19:04, Paul Bell  wrote:
>
> Hi All,
>
> I am running an nmap port scan on a Mesos agent node and noticed nmap
> reporting an open TCP port at 50577.
>
> Poking around some, I discovered exactly 5 mesos-docker-executor
> processes, one for each of my 5 Docker containers, and each with an open
> listen port:
>
> root 14131  3617  0 10:39 ?00:00:17 mesos-docker-executor
> --container=mesos-20151002-172703-2450482247-5050-3014-S0.5563c65a-e33e-4287-8ce4-b2aa8116aa95
> --docker=/usr/local/ecxmcc/weaveShim --help=false
> --mapped_directory=/mnt/mesos/sandbox
> --sandbox_directory=/tmp/mesos/slaves/20151002-172703-2450482247-5050-3014-S0/frameworks/20151002-172703-2450482247-5050-3014-/executors/postgres.ea2954fd-6b6e-11e5-8bef-56847afe9799/runs/5563c65a-e33e-4287-8ce4-b2aa8116aa95
> --stop_timeout=15secs
>
> I suppose that all of this is unsurprising. But I know of at least one big
> customer who will without delay run Nmap or Nessus against my clustered
> deployment.
>
> So I am wondering what the best practices approach is to securing these
> open ports.
>
> Thanks for your help.
>
> -Paul
>
>
>
>
>

Re: Scheduling tasks based on dependancy

[Guangya Liu]

Hi Pradeep,

>From here
https://github.com/apache/mesos/blob/master/src/master/master.cpp#L4913 you
can see that the offer does include resource attributes, so you can get the
filter the offers based on those attribute.

Thanks,

Guangya

On Tue, Oct 6, 2015 at 7:03 PM, Pradeep Kiruvale 
wrote:

> Hi Guangya,
>
> One doubt about the  --attributes=rackid:r1;groupid:g1 option.
>
> How does the master provisions the resources? How will be the resource
> offer?
>
> Is it like (Rack 1 , G1, System)? how does this way of  doing resource
> offer will help?
>
> Can you please give me more information?
>
>
> -Pradeep
>
>
>
> On 5 October 2015 at 17:45, Guangya Liu  wrote:
>
>> Hi Pradeep,
>>
>> I think that you can try Chronos and Marathon which can help you.
>>
>> *Marathon:* https://github.com/mesosphere/marathon
>> You can try Marathon + Mesos + Mesos Resource Attribute
>>
>> When you start up mesos slave, uses --attributes option, here is an
>> example:
>> ./bin/mesos-slave.sh --master=9.21.61.21:5050 --quiet
>> --log_dir=/tmp/mesos --attributes=rackid:r1;groupid:g1
>> This basically defines two attributes for this mesos slave host. rackid
>> with value r1 and groupid with value g1.
>>
>> marathon start -i "like_test" -C "sleep 100" -n 4 -c 1 -m 50 -o
>> "rackid:LIKE:r1"
>>
>> this will place applications on the slave node whose rackid is r1
>>
>> *Chronos:* https://github.com/mesos/chronos , Chronos supports the
>> definition of jobs triggered by the completion of other jobs. It supports
>> arbitrarily long dependency chains.
>>
>> Thanks,
>>
>> Guangya
>>
>> On Mon, Oct 5, 2015 at 11:21 PM, Pradeep Kiruvale <
>> pradeepkiruv...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Are there any frameworks that exists with the Mesos to schedule the
>>> bigger apps?
>>> I mean to say scheduling a app which has many services and will not fit
>>> into one physical node.
>>>
>>> Is there any frame work that can be used to
>>>  schedule tasks based on the underlying hardware constraints like
>>> Network bandwidth ?
>>>
>>  Schedule the tasks based on their dependencies and proximity to each
>>> other in a cluster or a rack?
>>>
>>> Thanks & Regards,
>>> Pradeep
>>>
>>
>>
>

Re: ARM64 version Mesos

[Michael Schenck]

out of curiosity, which ARM64 server(s) are you guys using?

On Tue, Oct 6, 2015 at 12:22 PM, haosdent  wrote:

> I think this issue maybe helpful for you.
> https://issues.apache.org/jira/browse/MESOS-2786 The patches maybe out of
> date.
>
> On Wed, Oct 7, 2015 at 12:13 AM, Pradeep Kiruvale <
> pradeepkiruv...@gmail.com> wrote:
>
>> Hi All,
>>
>> Is there a Mesos that runs on ARM64? I just tried compiling, its not
>> working, has some issues.
>>
>> Please let me know if some one already tried on ARM64. I am trying mesos
>> on a physical box, ubuntu running on it.
>>
>> Regards,
>> Pradeep
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: Scheduling tasks based on dependancy

[Sharma Podila]

Hi Pradeep,

We augment the mesos-slave command line on each agent to report the
available network bandwidth in Mbps. For example, the agents that have
1Gbps bandwidth have this additional custom resources (--resources command
line option) set: "network:1024". This shows up in the offers as a resource
with name "network" and value "1024", which can be used to assign. Similar
to how memory is assigned. That is, if you launch one task that requires
"network" resource of "100" value, that is, it is asking for 100 Mbps n/w
bandwidth, the next offer from Mesos will have network resource value of
924 (1024 - 100).
is this what you were asking for?

Sharma



On Tue, Oct 6, 2015 at 3:51 AM, Pradeep Kiruvale 
wrote:

> Hi Sharma,
>
> Is this how you collect the network info from the VMs?
>
> First you get the resource offers from the Mesos and then you collect the
> network bandwidth info and then you use that for assigning for your tasks?
> Or
> The mesos-slave collects the resource information? But I don't see any
> code to that and also the existing mesos-slave does not collects these
> resource information by itself.
>
> Am I missing something here?
>
> Regards,
> Pradeep
>
> On 5 October 2015 at 18:28, Sharma Podila  wrote:
>
>> Pradeep,
>>
>> We recently open sourced Fenzo  (wiki
>> ) to handle these scenarios. We
>> add a custom attribute for network bandwidth for each agent's "mesos-slave"
>> command line. And we have Fenzo assign resources to tasks based on CPU,
>> memory, disk, ports, and network bandwidth requirements. With Fenzo you can
>> define affinity, locality, and any other custom scheduling objectives using
>> plugins. Some of the plugins are already built in. It is also easy to add
>> additional plugins to cover other objectives you care about.
>>
>> "Dependencies" can mean multiple things. Do you mean dependencies on
>> certain attributes of resources/agents? Dependencies on where other tasks
>> are assigned? All of these are covered. However, if you mean workflow type
>> of dependencies on completion of other tasks, then, there are no built in
>> plugins. You could write one using Fenzo. It is also common for such
>> workflow dependencies to be covered by an entity external to the scheduler.
>> Both techniques can be made to work.
>>
>> Fenzo has the concept of hard Vs soft constraints. You could specify, for
>> example, resource affinity and/or task locality as a soft constraint or a
>> hard constraint. See the wiki docs link I provided above for details.
>>
>> Sharma
>>
>>
>> On Mon, Oct 5, 2015 at 8:21 AM, Pradeep Kiruvale <
>> pradeepkiruv...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Are there any frameworks that exists with the Mesos to schedule the
>>> bigger apps?
>>> I mean to say scheduling a app which has many services and will not fit
>>> into one physical node.
>>>
>>> Is there any frame work that can be used to
>>>  schedule tasks based on the underlying hardware constraints like
>>> Network bandwidth ?
>>>  Schedule the tasks based on their dependencies and proximity to each
>>> other in a cluster or a rack?
>>>
>>> Thanks & Regards,
>>> Pradeep
>>>
>>
>>
>

Re: Scheduling tasks based on dependancy

[Sharma Podila]

Pradeep, attributes show up as name value pairs in the offers. Custom
attributes can also be used in Fenzo for assignment optimizations. For
example, we set custom attributes for AWS EC2 ZONE names and ASG names. We
use the ZONE name custom attribute to balance tasks of a job across zones
via the built in constraint plugin, BalancedHostAttributeConstraint



On Tue, Oct 6, 2015 at 4:03 AM, Pradeep Kiruvale 
wrote:

> Hi Guangya,
>
> One doubt about the  --attributes=rackid:r1;groupid:g1 option.
>
> How does the master provisions the resources? How will be the resource
> offer?
>
> Is it like (Rack 1 , G1, System)? how does this way of  doing resource
> offer will help?
>
> Can you please give me more information?
>
>
> -Pradeep
>
>
>
> On 5 October 2015 at 17:45, Guangya Liu  wrote:
>
>> Hi Pradeep,
>>
>> I think that you can try Chronos and Marathon which can help you.
>>
>> *Marathon:* https://github.com/mesosphere/marathon
>> You can try Marathon + Mesos + Mesos Resource Attribute
>>
>> When you start up mesos slave, uses --attributes option, here is an
>> example:
>> ./bin/mesos-slave.sh --master=9.21.61.21:5050 --quiet
>> --log_dir=/tmp/mesos --attributes=rackid:r1;groupid:g1
>> This basically defines two attributes for this mesos slave host. rackid
>> with value r1 and groupid with value g1.
>>
>> marathon start -i "like_test" -C "sleep 100" -n 4 -c 1 -m 50 -o
>> "rackid:LIKE:r1"
>>
>> this will place applications on the slave node whose rackid is r1
>>
>> *Chronos:* https://github.com/mesos/chronos , Chronos supports the
>> definition of jobs triggered by the completion of other jobs. It supports
>> arbitrarily long dependency chains.
>>
>> Thanks,
>>
>> Guangya
>>
>> On Mon, Oct 5, 2015 at 11:21 PM, Pradeep Kiruvale <
>> pradeepkiruv...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Are there any frameworks that exists with the Mesos to schedule the
>>> bigger apps?
>>> I mean to say scheduling a app which has many services and will not fit
>>> into one physical node.
>>>
>>> Is there any frame work that can be used to
>>>  schedule tasks based on the underlying hardware constraints like
>>> Network bandwidth ?
>>>
>>  Schedule the tasks based on their dependencies and proximity to each
>>> other in a cluster or a rack?
>>>
>>> Thanks & Regards,
>>> Pradeep
>>>
>>
>>
>

Re: ARM64 version Mesos

[haosdent]

I think this issue maybe helpful for you.
https://issues.apache.org/jira/browse/MESOS-2786 The patches maybe out of
date.

On Wed, Oct 7, 2015 at 12:13 AM, Pradeep Kiruvale  wrote:

> Hi All,
>
> Is there a Mesos that runs on ARM64? I just tried compiling, its not
> working, has some issues.
>
> Please let me know if some one already tried on ARM64. I am trying mesos
> on a physical box, ubuntu running on it.
>
> Regards,
> Pradeep
>



-- 
Best Regards,
Haosdent Huang

Re: Securing executors

[Adam Bordelon]

Paul, yes encryption is a possibility since Mesos 0.23. See
http://mesos.apache.org/documentation/latest/mesos-ssl/
I believe you can also select which listener port you want to use by
specifying LIBPROCESS_PORT in the executor's environment.

On Tue, Oct 6, 2015 at 6:59 AM, Paul Bell  wrote:

> Thanks, Alexander; I will check out the vid.
>
> I kind of assumed that this port was used for exactly the purpose you
> mention.
>
> Is TLS a possibility here?
>
> -Paul
>
> On Tue, Oct 6, 2015 at 8:15 AM, Alexander Rojas 
> wrote:
>
>> Hi Paul,
>>
>> I can refer you to the talk given by Adam Bordelon at MesosCon
>> https://www.youtube.com/watch?v=G3sn1OLYDOE
>>
>> If you want to the short answer, the solution is to put a firewall around
>> your cluster.
>>
>> On a closer look on the port, it is the one used for message passing
>> between the mesas-docker-executor and other mesos components.
>>
>>
>> On 05 Oct 2015, at 19:04, Paul Bell  wrote:
>>
>> Hi All,
>>
>> I am running an nmap port scan on a Mesos agent node and noticed nmap
>> reporting an open TCP port at 50577.
>>
>> Poking around some, I discovered exactly 5 mesos-docker-executor
>> processes, one for each of my 5 Docker containers, and each with an open
>> listen port:
>>
>> root 14131  3617  0 10:39 ?00:00:17 mesos-docker-executor
>> --container=mesos-20151002-172703-2450482247-5050-3014-S0.5563c65a-e33e-4287-8ce4-b2aa8116aa95
>> --docker=/usr/local/ecxmcc/weaveShim --help=false
>> --mapped_directory=/mnt/mesos/sandbox
>> --sandbox_directory=/tmp/mesos/slaves/20151002-172703-2450482247-5050-3014-S0/frameworks/20151002-172703-2450482247-5050-3014-/executors/postgres.ea2954fd-6b6e-11e5-8bef-56847afe9799/runs/5563c65a-e33e-4287-8ce4-b2aa8116aa95
>> --stop_timeout=15secs
>>
>> I suppose that all of this is unsurprising. But I know of at least one
>> big customer who will without delay run Nmap or Nessus against my clustered
>> deployment.
>>
>> So I am wondering what the best practices approach is to securing these
>> open ports.
>>
>> Thanks for your help.
>>
>> -Paul
>>
>>
>>
>>
>>
>

ARM64 version Mesos

[Pradeep Kiruvale]

Hi All,

Is there a Mesos that runs on ARM64? I just tried compiling, its not
working, has some issues.

Please let me know if some one already tried on ARM64. I am trying mesos on
a physical box, ubuntu running on it.

Regards,
Pradeep

Re: ARM64 version Mesos

[Pradeep Kiruvale]

As of now I am using Applied Micro box.
We also have AMD (Seatle) box, Cavium Thunder X and our own home grown ARM
boxes.

-Pradeep

On 6 October 2015 at 18:24, Michael Schenck  wrote:

> out of curiosity, which ARM64 server(s) are you guys using?
>
> On Tue, Oct 6, 2015 at 12:22 PM, haosdent  wrote:
>
>> I think this issue maybe helpful for you.
>> https://issues.apache.org/jira/browse/MESOS-2786 The patches maybe out
>> of date.
>>
>> On Wed, Oct 7, 2015 at 12:13 AM, Pradeep Kiruvale <
>> pradeepkiruv...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Is there a Mesos that runs on ARM64? I just tried compiling, its not
>>> working, has some issues.
>>>
>>> Please let me know if some one already tried on ARM64. I am trying mesos
>>> on a physical box, ubuntu running on it.
>>>
>>> Regards,
>>> Pradeep
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>