Re: cgroup limits on memory page cache

[Charles Allen]

Of course, after sending out the email I thought of a potential solution.
It should be possible to specify memory available per role that is more
than the total system memory. So as long as the role understands that a
large swath of the available memory is intended to be page cache, all
should be good... I think

On Tue, Aug 9, 2016 at 8:16 PM Charles Allen 
wrote:

> I have two services I want to make co-tenant on the same machines, where
> one has much more relaxed SLAs than the other.
>
> I can easily make one pre-empt the other at a cgroup level by controlling
> their cpu shares.
>
> And I can limit one by having memory limits assignable through its role
> resources.
>
> The other higher priority service makes heavy use of disk page cache, and
> therefore would like to use as much memory for disk page cache as possible,
> but allow the lower-priority service to gobble up SOME of that page cache
> (to a limit)
>
> This is achievable with cgroups alone by setting appropriate cpu shares
> between the cgroups, and setting the memory limit of one cgroup to
> unlimited, while bounding the memory limit of the lesser priority service.
>
> Can I achieve such a thing in Mesos role resource definitions?
>

cgroup limits on memory page cache

[Charles Allen]

I have two services I want to make co-tenant on the same machines, where
one has much more relaxed SLAs than the other.

I can easily make one pre-empt the other at a cgroup level by controlling
their cpu shares.

And I can limit one by having memory limits assignable through its role
resources.

The other higher priority service makes heavy use of disk page cache, and
therefore would like to use as much memory for disk page cache as possible,
but allow the lower-priority service to gobble up SOME of that page cache
(to a limit)

This is achievable with cgroups alone by setting appropriate cpu shares
between the cgroups, and setting the memory limit of one cgroup to
unlimited, while bounding the memory limit of the lesser priority service.

Can I achieve such a thing in Mesos role resource definitions?

Re: Initial Design Document Apache Mesos Federation (JIRA 3548)

[DhilipKumar Sankaranarayanan]

Hi ,

In the proposed the design the PE(Policy Engine) will recommend a new DC
the framework can connect to at the time of bursting.  It is upto the
framework to perform contraction or suppress itself from the ' public cloud
' and use offers from its own datacenter.

Yes the second part is true, the DC to be preferred for what workload is
entirely upto the Policy Engine's implementation.  The idea is that
framework will now connect to multiple Mesos Masters and receive offers
from them at the same time.

Which Master it will burst to  is decided by the PE, how to deal with the
offers after connecting to the master is decided by the Framework itself.

Hi All,

A quick update on the survey, so far 25 have responded and 13 of them
prefer Hierarchical.  Its good to know that 52% of them prefer not to
change the framework much, where as 48% of them dont mind changing the
framework code reasonably to connect multiple masters directly.  I hope
this gets the Mesos committers attention to progress further in this
project. :-)

[image: Inline image 1]

On Sun, Aug 7, 2016 at 10:19 AM, Lawrence Rau  wrote:

> if you “burst” into another datacenter because of unavailable “local”
> resources do you contract when resources are free?  If your purpose of
> bursting is the use case of an on-prem site bursting into a public provider
> it would seem logical the contraction would be desired to free that public
> (and presumably more costly) resources are released.  This is amplified
> perhaps if you end up in multiple DC’s if on each local resource contention
> a different “foreign” cluster is chosen to burst into — unless you further
> plan to bias towards keeping your “burst tasks” in a same DC if possible
> (e.g. given DC-{A,B,C} once you burst from DC-A to DC-B you’d prefer B over
> C for future task launches) or maybe you don’t care (or maybe this is a
> policy?)
>
>
>
> On Jul 15, 2016, at 6:46 PM, DhilipKumar Sankaranarayanan <
> s.dhilipku...@gmail.com> wrote:
>
> Hi All,
>
> I got a chance to bring this up during yesterdays Community Sync.  It was
> great discussing with you all.
>
> As a general feedback the role of policy engine in the design needs to be
> clearer, i will update the Document with more information on PE very soon.
>
> We are yet to get more insight on the License issues like bringing in a
> Mozzilla 2.0 library into an Apache 2.0 project.
>
> It will be fantastic to get more thoughts on this from the community so
> please share if you or your organisation had thought about it.
>
> HI Alex,
>
> Thanks again.
>
> a) Yes you are correct, thats exactly what we thought, a Framework could
> simply query and learn about its next step (bursting or load balancing).
> b)  We are currently thinking that the Framework will run in only one
> place and should be able to connect to other datacenters.  Each data
> centres could have some Frameworks running the local and some part of a
> federation.
>
> Regards,
> Dhilip
>
>
> On Thu, Jul 14, 2016 at 9:17 AM, Alexander Gallego 
> wrote:
>
>>
>>
>> On Thu, Jul 14, 2016 at 2:40 AM, DhilipKumar Sankaranarayanan <
>> s.dhilipku...@gmail.com> wrote:
>>
>>> HI Alex,
>>>
>>> Thanks for taking a look.  We have simplified the design since the
>>> conference.  The Allocation and Anonymous modules where only helping us to
>>> control the offers sent to the frameworks.  Now we think that Roles and
>>> Quota in Moses elegantly solve this problem and we could take advantage of
>>> it.
>>>
>>
>> Sounds good, given that the design is entirely different now, can you
>> share some of these thoughts.
>>
>>
>>>
>>> The current design does not propose Mesos Modules, the POC we
>>> demonstrated @ the mesoscon is slightly out of date in that respect.
>>>
>>> The current design only enforces that any Policy Engine implementation
>>> should honour certain REST apis.   This also removes Consul out of the
>>> picture, but at Huawei our implementation would pretty much consider Consul
>>> or something similar.
>>>
>>> 1) Failure semantics
>>> I do agree it is not straight forward to declare that a DC is lost just
>>> because framework lost the connection intermittently.  Probing the
>>> 'Gossiper' we would know that the DC is still active but not just reachable
>>> to us,  In that case its worth the wait.  If the DC in question is not
>>> reachable from everyother DC, only then we could come to such conclusion.
>>>
>>>
>>
>> how do you envision frameworks integrating w/ this. Are you saying that
>> frameworks should poll the HTTP endpoint of the Gossiper?
>>
>>
>>
>>> 2)  Can you share more details about the allocator modules.
>>> As mentioned earlier these modules are no longer relevant we have much
>>> simpler way to achieve this.
>>>
>>> 3) High Availability
>>> I think you are talking about the below section?
>>> "Sequence Diagram for High Availability
>>> (Incase of local datacenter failure)
>>> Very Similar to cloud bursting use-case scenario.  "

Re: 1.0.1 release

[Benjamin Mahler]

All of the issues I've been shepherding have been fixed.

The only one I see remaining is this one, but doesn't look like a blocking
issue: https://issues.apache.org/jira/browse/MESOS-5985

Anything else that needs to go in?

On Mon, Aug 1, 2016 at 4:19 PM, Vinod Kone  wrote:

> Hi,
>
> As discussed on the 1.0 voting thread, we plan to cut a 1.0.1 as early as
> this week. So if you have anything that needs to absolutely go into the
> patch release, please work with your shepherd and get it landed on trunk
> and backported to the 1.0.x branch.
>
> Thanks,
>

The External Containerizer is Removed

[Gilbert Song]

Hello All,

FYI, as we discussed on the mailing list in Oct 2015 (link
)
and Apr 2016 (link

),
the External Containerizer was deprecated.

Now, we removed the External Containerizer from Mesos. Please see
MESOS-3370  for details.

Gilbert

Re: How could Mesos work with docker?

[Gilbert Song]

Yu,

Mesos supports docker images without using the docker daemon. This feature
is called Unified Containerizer, which was introduced since Mesos 0.28.
Please
follow this document to launch your containers:

http://mesos.apache.org/documentation/latest/container-image/

Feel free to join our slack channel 
if you have any question.

Gilbert

On Tue, Aug 9, 2016 at 7:13 AM, Yu Wei  wrote:

> Hi Tomek,
>
>
> Good point.
> Thanks very much for your help.
>
>
> Jared, (韦煜）
> Software developer
> Interested in open source software, big data, Linux
> --
> *From:* Tomek Janiszewski 
> *Sent:* Tuesday, August 9, 2016 10:06:28 PM
> *To:* d...@mesos.apache.org; user@mesos.apache.org
> *Subject:* Re: How could Mesos work with docker?
>
> Hi
>
> Docker support is described here http://mesos.apache.org/
> documentation/latest/docker-containerizer/
>
> Best
> Tomek
>
> wt., 9.08.2016 o 16:04 użytkownik Yu Wei  napisał:
>
>> Hi guys,
>>
>> I start learning mesos.
>>
>> How could mesos work with docker? Is there any document about this?
>>
>>
>> Any other advice?
>>
>>
>> Thx,
>>
>>
>> Jared, (韦煜）
>> Software developer
>> Interested in open source software, big data, Linux
>>
>

Re: what's the pronunciation of "MESOS"?

[Stephen Boesch]

@Jared / Yu Wei:  Mesos is essentially a spanish word: so MAY-sos  would
travel well.

2016-08-09 11:35 GMT-07:00 Ken Sipe :

> Apparently it depends on if you are British or not :)
> http://dictionary.cambridge.org/us/pronunciation/english/the-mesosphere
>
> apparently the absence of “phere" changes everything:
> https://www.howtopronounce.com/mesos/
>
> and for this looking for which percentile they are in:
> http://www.basilmarket.com/How-do-You-pronounce-mesos-Thread-b5eAL-1
>
>
> On Aug 9, 2016, at 12:30 PM, Charles Allen 
> wrote:
>
> My wife thought I was crazy sitting here mumbling "mAY-sohs" "MEH-sohs"
> "Mee-sohs"
>
> On Mon, Aug 8, 2016 at 9:22 PM Yu Wei  wrote:
>
>> Thanks Joe.
>>
>> It's really interesting.
>>
>>
>> Jared, (韦煜）
>> Software developer
>> Interested in open source software, big data, Linux
>>
>>
>> --
>> *From:* Joseph Jacks 
>> *Sent:* Tuesday, August 9, 2016 10:53 AM
>> *To:* user@mesos.apache.org
>> *Subject:* Re: what's the pronunciation of "MESOS"?
>>
>> "MAY-zoss" is most common and correct.
>>
>> "MEH-zoss" is second most common and also correct, I think.
>>
>> "MEE-zoss" is third most common, but incorrect.
>>
>> JJ.
>>
>> On Aug 8, 2016, at 10:48 PM, Yu Wei  wrote:
>>
>>
>> Thx,
>>
>> Jared, (韦煜）
>> Software developer
>> Interested in open source software, big data, Linux
>>
>>
>

Re: what's the pronunciation of "MESOS"?

[Ken Sipe]

Apparently it depends on if you are British or not :)   
http://dictionary.cambridge.org/us/pronunciation/english/the-mesosphere 


apparently the absence of “phere" changes everything:  
https://www.howtopronounce.com/mesos/ 

and for this looking for which percentile they are in:  
http://www.basilmarket.com/How-do-You-pronounce-mesos-Thread-b5eAL-1 



> On Aug 9, 2016, at 12:30 PM, Charles Allen  
> wrote:
> 
> My wife thought I was crazy sitting here mumbling "mAY-sohs" "MEH-sohs" 
> "Mee-sohs"
> 
> On Mon, Aug 8, 2016 at 9:22 PM Yu Wei  > wrote:
> Thanks Joe.
> 
> It's really interesting.
> 
> Jared, (韦煜）
> Software developer
> Interested in open source software, big data, Linux
> 
> 
> From: Joseph Jacks >
> Sent: Tuesday, August 9, 2016 10:53 AM
> To: user@mesos.apache.org 
> Subject: Re: what's the pronunciation of "MESOS"?
>  
> "MAY-zoss" is most common and correct. 
> 
> "MEH-zoss" is second most common and also correct, I think. 
> 
> "MEE-zoss" is third most common, but incorrect.
> 
> JJ.
> 
> On Aug 8, 2016, at 10:48 PM, Yu Wei  > wrote:
> 
>> 
>> Thx,
>> 
>> Jared, (韦煜）
>> Software developer
>> Interested in open source software, big data, Linux

Re: Master pailer failure 0.28.2

[haosdent]

Hi, @Charles Allen I use 1.0.0 and it looks fine in my master. Does it
happen after you refresh?

On Wed, Aug 10, 2016 at 1:31 AM, Charles Allen <
charles.al...@metamarkets.com> wrote:

> Slave pailer is working fine
>
> On Tue, Aug 9, 2016 at 10:29 AM Charles Allen <
> charles.al...@metamarkets.com> wrote:
>
>> For some reason I started getting the following failure on 0.28.2 with
>> the pailer when trying to view master logs from the master console
>> (REDACTED is the ip address):
>>
>>
>> angular-1.2.3.min.js:84 Error: [$interpolate:interr]
>> http://errors.angularjs.org/1.2.3/$interpolate/interr?p0=%
>> 7B%7Boffered_cpus…7D=TypeError%3A%20Cannot%20read%
>> 20property%20'toFixed'%20of%20undefined
>> at Error (native)
>> at http://REDACTED:5050/static/js/angular-1.2.3.min.js:6:449
>> at Object.s (http://REDACTED:5050/static/
>> js/angular-1.2.3.min.js:73:495)
>> at f.$digest (http://REDACTED:5050/static/
>> js/angular-1.2.3.min.js:99:14)
>> at f.$apply (http://REDACTED:5050/static/js/angular-1.2.3.min.js:101:
>> 369)
>> at f (http://REDACTED:5050/static/js/angular-1.2.3.min.js:67:175)
>> at Q (http://REDACTED:5050/static/js/angular-1.2.3.min.js:71:99)
>> at XMLHttpRequest.y.onreadystatechange (http://REDACTED:5050/static/
>> js/angular-1.2.3.min.js:72:130)(anonymous function) @
>> angular-1.2.3.min.js:84(anonymous function) @ angular-1.2.3.min.js:62s @
>> angular-1.2.3.min.js:74$digest @ angular-1.2.3.min.js:99$apply @
>> angular-1.2.3.min.js:101f @ angular-1.2.3.min.js:67Q @
>> angular-1.2.3.min.js:71y.onreadystatechange @ angular-1.2.3.min.js:72
>> angular-1.2.3.min.js:84 Error: [$interpolate:interr]
>> http://errors.angularjs.org/1.2.3/$interpolate/interr?p0=%
>> 7B%7Bidle_cpus%20…7D=TypeError%3A%20Cannot%20read%
>> 20property%20'toFixed'%20of%20undefined
>> at Error (native)
>> at http://REDACTED:5050/static/js/angular-1.2.3.min.js:6:449
>> at Object.s (http://REDACTED:5050/static/
>> js/angular-1.2.3.min.js:73:495)
>> at f.$digest (http://REDACTED:5050/static/
>> js/angular-1.2.3.min.js:99:14)
>> at f.$apply (http://REDACTED:5050/static/js/angular-1.2.3.min.js:101:
>> 369)
>> at f (http://REDACTED:5050/static/js/angular-1.2.3.min.js:67:175)
>> at Q (http://REDACTED:5050/static/js/angular-1.2.3.min.js:71:99)
>> at XMLHttpRequest.y.onreadystatechange (http://REDACTED:5050/static/
>> js/angular-1.2.3.min.js:72:130)(anonymous function) @
>> angular-1.2.3.min.js:84(anonymous function) @ angular-1.2.3.min.js:62s @
>> angular-1.2.3.min.js:74$digest @ angular-1.2.3.min.js:99$apply @
>> angular-1.2.3.min.js:101f @ angular-1.2.3.min.js:67Q @
>> angular-1.2.3.min.js:71y.onreadystatechange @ angular-1.2.3.min.js:72
>>
>>
>> Has anyone seen this before?
>>
>


-- 
Best Regards,
Haosdent Huang

Re: Master pailer failure 0.28.2

[Charles Allen]

Slave pailer is working fine

On Tue, Aug 9, 2016 at 10:29 AM Charles Allen 
wrote:

> For some reason I started getting the following failure on 0.28.2 with the
> pailer when trying to view master logs from the master console (REDACTED is
> the ip address):
>
>
> angular-1.2.3.min.js:84 Error: [$interpolate:interr]
> http://errors.angularjs.org/1.2.3/$interpolate/interr?p0=%7B%7Boffered_cpus
> …7D=TypeError%3A%20Cannot%20read%20property%20'toFixed'%20of%20undefined
> at Error (native)
> at http://REDACTED:5050/static/js/angular-1.2.3.min.js:6:449
> at Object.s (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:73:495)
> at f.$digest (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:99:14)
> at f.$apply (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:101:369)
> at f (http://REDACTED:5050/static/js/angular-1.2.3.min.js:67:175)
> at Q (http://REDACTED:5050/static/js/angular-1.2.3.min.js:71:99)
> at XMLHttpRequest.y.onreadystatechange (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:72:130)(anonymous
> function) @ angular-1.2.3.min.js:84(anonymous function) @
> angular-1.2.3.min.js:62s @ angular-1.2.3.min.js:74$digest @
> angular-1.2.3.min.js:99$apply @ angular-1.2.3.min.js:101f @
> angular-1.2.3.min.js:67Q @ angular-1.2.3.min.js:71y.onreadystatechange @
> angular-1.2.3.min.js:72
> angular-1.2.3.min.js:84 Error: [$interpolate:interr]
> http://errors.angularjs.org/1.2.3/$interpolate/interr?p0=%7B%7Bidle_cpus%20
> …7D=TypeError%3A%20Cannot%20read%20property%20'toFixed'%20of%20undefined
> at Error (native)
> at http://REDACTED:5050/static/js/angular-1.2.3.min.js:6:449
> at Object.s (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:73:495)
> at f.$digest (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:99:14)
> at f.$apply (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:101:369)
> at f (http://REDACTED:5050/static/js/angular-1.2.3.min.js:67:175)
> at Q (http://REDACTED:5050/static/js/angular-1.2.3.min.js:71:99)
> at XMLHttpRequest.y.onreadystatechange (
> http://REDACTED:5050/static/js/angular-1.2.3.min.js:72:130)(anonymous
> function) @ angular-1.2.3.min.js:84(anonymous function) @
> angular-1.2.3.min.js:62s @ angular-1.2.3.min.js:74$digest @
> angular-1.2.3.min.js:99$apply @ angular-1.2.3.min.js:101f @
> angular-1.2.3.min.js:67Q @ angular-1.2.3.min.js:71y.onreadystatechange @
> angular-1.2.3.min.js:72
>
>
> Has anyone seen this before?
>

Re: what's the pronunciation of "MESOS"?

[Charles Allen]

My wife thought I was crazy sitting here mumbling "mAY-sohs" "MEH-sohs"
"Mee-sohs"

On Mon, Aug 8, 2016 at 9:22 PM Yu Wei  wrote:

> Thanks Joe.
>
> It's really interesting.
>
>
> Jared, (韦煜）
> Software developer
> Interested in open source software, big data, Linux
>
>
> --
> *From:* Joseph Jacks 
> *Sent:* Tuesday, August 9, 2016 10:53 AM
> *To:* user@mesos.apache.org
> *Subject:* Re: what's the pronunciation of "MESOS"?
>
> "MAY-zoss" is most common and correct.
>
> "MEH-zoss" is second most common and also correct, I think.
>
> "MEE-zoss" is third most common, but incorrect.
>
> JJ.
>
> On Aug 8, 2016, at 10:48 PM, Yu Wei  wrote:
>
>
> Thx,
>
> Jared, (韦煜）
> Software developer
> Interested in open source software, big data, Linux
>
>

Master pailer failure 0.28.2

[Charles Allen]

For some reason I started getting the following failure on 0.28.2 with the
pailer when trying to view master logs from the master console (REDACTED is
the ip address):


angular-1.2.3.min.js:84 Error: [$interpolate:interr]
http://errors.angularjs.org/1.2.3/$interpolate/interr?p0=%7B%7Boffered_cpus
…7D=TypeError%3A%20Cannot%20read%20property%20'toFixed'%20of%20undefined
at Error (native)
at http://REDACTED:5050/static/js/angular-1.2.3.min.js:6:449
at Object.s (http://REDACTED:5050/static/js/angular-1.2.3.min.js:73:495)
at f.$digest (http://REDACTED:5050/static/js/angular-1.2.3.min.js:99:14)
at f.$apply (http://REDACTED:5050/static/js/angular-1.2.3.min.js:101:369
)
at f (http://REDACTED:5050/static/js/angular-1.2.3.min.js:67:175)
at Q (http://REDACTED:5050/static/js/angular-1.2.3.min.js:71:99)
at XMLHttpRequest.y.onreadystatechange (
http://REDACTED:5050/static/js/angular-1.2.3.min.js:72:130)(anonymous
function) @ angular-1.2.3.min.js:84(anonymous function) @
angular-1.2.3.min.js:62s @ angular-1.2.3.min.js:74$digest @
angular-1.2.3.min.js:99$apply @ angular-1.2.3.min.js:101f @
angular-1.2.3.min.js:67Q @ angular-1.2.3.min.js:71y.onreadystatechange @
angular-1.2.3.min.js:72
angular-1.2.3.min.js:84 Error: [$interpolate:interr]
http://errors.angularjs.org/1.2.3/$interpolate/interr?p0=%7B%7Bidle_cpus%20
…7D=TypeError%3A%20Cannot%20read%20property%20'toFixed'%20of%20undefined
at Error (native)
at http://REDACTED:5050/static/js/angular-1.2.3.min.js:6:449
at Object.s (http://REDACTED:5050/static/js/angular-1.2.3.min.js:73:495)
at f.$digest (http://REDACTED:5050/static/js/angular-1.2.3.min.js:99:14)
at f.$apply (http://REDACTED:5050/static/js/angular-1.2.3.min.js:101:369
)
at f (http://REDACTED:5050/static/js/angular-1.2.3.min.js:67:175)
at Q (http://REDACTED:5050/static/js/angular-1.2.3.min.js:71:99)
at XMLHttpRequest.y.onreadystatechange (
http://REDACTED:5050/static/js/angular-1.2.3.min.js:72:130)(anonymous
function) @ angular-1.2.3.min.js:84(anonymous function) @
angular-1.2.3.min.js:62s @ angular-1.2.3.min.js:74$digest @
angular-1.2.3.min.js:99$apply @ angular-1.2.3.min.js:101f @
angular-1.2.3.min.js:67Q @ angular-1.2.3.min.js:71y.onreadystatechange @
angular-1.2.3.min.js:72


Has anyone seen this before?

Re: Mesos logging

[Hendrik Haddorp]

I saw a few "Running ..." log entries from the docker support code but
they seem to be all from VLOG(1) calls while for some reason the code
that does the actual "docker run" call uses LOG(INFO) and that does not
seem to come out by default, or I don't see it. But I'll try on ...

On 09/08/16 11:38, haosdent wrote:
> Hi, @Hendrik You could see INFO log when running Mesos Agent in
> default level. Some docker run logs may exist in the stdout/stderr of
> executor.
>
> On Tue, Aug 9, 2016 at 12:27 PM, Hendrik Haddorp
> > wrote:
>
> I would like to see the "docker run" trace from
> https://github.com/apache/mesos/blob/master/src/docker/docker.cpp
> 
> line 811.
> What verbosity level does INFO map to?
>
> On 09/08/16 05:06, Charles Allen wrote:
> > Which glog are you trying to capture? You can set the verbosity
> level
> > with the environment variable GLOG_v
> >
> > And you can also set it through things like Spark. So if you want a
> > lot of ZK chatter at the mesos level in your spark logs, add
> >
> > spark.executorEnv.GLOG_v=9
> >
> > to your spark context
> >
> > On Mon, Aug 8, 2016 at 2:53 PM Hendrik Haddorp
> > 
> >>
> wrote:
> >
> > Hi,
> >
> > the Mesos code contains log statements using LOG(INFO) and
> > VLOG(1), for
> > example. So far I found that Mesos is using the Google Logging
> > Library.
> > Looking in the logs I only seem to be able to find output
> from VLOG
> > statements. What do I need to do to get the output from the LOG
> > statements? Where would I typically find the output? I'm
> using CentOS
> > 7.2 and found the output so far in the files below
> /var/log/mesos.
> >
> > thanks,
> > Hendrik
> >
>
>
>
>
> -- 
> Best Regards,
> Haosdent Huang

Re: How could Mesos work with docker?

[Yu Wei]

Hi Tomek,


Good point.
Thanks very much for your help.


Jared, (韦煜）
Software developer
Interested in open source software, big data, Linux


From: Tomek Janiszewski 
Sent: Tuesday, August 9, 2016 10:06:28 PM
To: d...@mesos.apache.org; user@mesos.apache.org
Subject: Re: How could Mesos work with docker?

Hi

Docker support is described here 
http://mesos.apache.org/documentation/latest/docker-containerizer/

Best
Tomek

wt., 9.08.2016 o 16:04 u?ytkownik Yu Wei 
> napisa?:

Hi guys,

I start learning mesos.

How could mesos work with docker? Is there any document about this?


Any other advice?


Thx,


Jared, (韦煜）
Software developer
Interested in open source software, big data, Linux

Re: How could Mesos work with docker?

[Tomek Janiszewski]

Hi

Docker support is described here
http://mesos.apache.org/documentation/latest/docker-containerizer/

Best
Tomek

wt., 9.08.2016 o 16:04 użytkownik Yu Wei  napisał:

> Hi guys,
>
> I start learning mesos.
>
> How could mesos work with docker? Is there any document about this?
>
>
> Any other advice?
>
>
> Thx,
>
>
> Jared, (韦煜）
> Software developer
> Interested in open source software, big data, Linux
>

How could Mesos work with docker?

[Yu Wei]

Hi guys,

I start learning mesos.

How could mesos work with docker? Is there any document about this?


Any other advice?


Thx,


Jared, (??)
Software developer
Interested in open source software, big data, Linux

RE: Support for tasks groups aka pods in Mesos

[Aaron Carey]

Just had a brief look over, this is great, a huge leap forward.. we were 
considering moving to Kubernetes because of the pod support, having this in 
mesos would allow us to do a lot more!


--

Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150


From: Vinod Kone [vinodk...@apache.org]
Sent: 09 August 2016 02:42
To: Vinod Kone
Cc: dev; user
Subject: Re: Support for tasks groups aka pods in Mesos

Sorry, sent the wrong link earlier for design doc.

Design doc: https://issues.apache.org/jira/browse/MESOS-6009

Direct link: 
https://docs.google.com/document/d/1FtcyQkDfGp-bPHTW4pUoqQCgVlPde936bo-IIENO_ho/edit#heading=h.ip4t59nlogfz

答复: Deactivationg framework unexpectly

[志昌 余]

Hi Anindys,

Thanks for the info. I'll enable  scheduler driver log to see what happen.

Regards,

Zhichang Yu


发件人: anindya_si...@apple.com  代表 Anindya Sinha 

发送时间: 2016年8月8日 23:50:10
收件人: user@mesos.apache.org
主题: Re: Deactivationg framework unexpectly

Looks like your framework (chronos) is sending a DeactivateFrameworkMessage 
message to the master. The scheduler driver would also send a 
DeativateFramework message if it is aborted 
(https://github.com/apache/mesos/blob/master/src/sched/sched.cpp#L1224).

Also, master can deactivate your framework if your framework disconnects or 
fails over. Please check logs in master or see if your framework received a 
FrameworkErrorMessage.

Thanks
Anindya

On Aug 8, 2016, at 3:35 AM, 志昌 余 
> wrote:

Hi,
I recently faced a wired problem. I'm running mesos + chronos. Chronos 
often (once every several days) stops scheduling tasks due to mesos deactived 
the framework.
As following is the log of mesos master leader:


# grep -iP "activat|disconnected" /var/log/mesos/mesos-master.INFO
I0806 13:40:33.14365830 master.cpp:2551] Deactivating framework 
90a6a7dc-7256-4e55-bd7e-573233c5df74- (chronos-2.5.0-SNAPSHOT) at 
scheduler-86a64d22-5201-4bb0-8a2c-70d3e97afae6@10.8.139.246:34544
I0806 13:40:33.14390823 hierarchical.cpp:375] Deactivated framework 
90a6a7dc-7256-4e55-bd7e-573233c5df74-

The fix is to manually reboot the chronos leader.


My env:
There are 3 physical machines, on each are running containerized mesos master 
and chronos. When the issue occurred,  the mesos leader and chronos leader were 
both running on the same machine.

Software Version:
mesos-master:0.28.0-2.0.16.ubuntu1404

chronos:2.5.0-ce4469d.ubuntu1404-mesos-0.28.0-2.0.16.ubuntu1404

Can anyone give insight for this problem?
Thanks,
Zhichang Yu

Re: Mesos logging

[haosdent]

Hi, @Hendrik You could see INFO log when running Mesos Agent in default
level. Some docker run logs may exist in the stdout/stderr of executor.

On Tue, Aug 9, 2016 at 12:27 PM, Hendrik Haddorp 
wrote:

> I would like to see the "docker run" trace from
> https://github.com/apache/mesos/blob/master/src/docker/docker.cpp line
> 811.
> What verbosity level does INFO map to?
>
> On 09/08/16 05:06, Charles Allen wrote:
> > Which glog are you trying to capture? You can set the verbosity level
> > with the environment variable GLOG_v
> >
> > And you can also set it through things like Spark. So if you want a
> > lot of ZK chatter at the mesos level in your spark logs, add
> >
> > spark.executorEnv.GLOG_v=9
> >
> > to your spark context
> >
> > On Mon, Aug 8, 2016 at 2:53 PM Hendrik Haddorp
> > > wrote:
> >
> > Hi,
> >
> > the Mesos code contains log statements using LOG(INFO) and
> > VLOG(1), for
> > example. So far I found that Mesos is using the Google Logging
> > Library.
> > Looking in the logs I only seem to be able to find output from VLOG
> > statements. What do I need to do to get the output from the LOG
> > statements? Where would I typically find the output? I'm using CentOS
> > 7.2 and found the output so far in the files below /var/log/mesos.
> >
> > thanks,
> > Hendrik
> >
>
>


-- 
Best Regards,
Haosdent Huang