Re: Welcome Neil Conway as Mesos Committer and PMC member!

Congrats Neil!! On Sat, 21 Jan 2017 at 12:34 Vinod Kone wrote: > Hi folks, > > Please welcome Neil Conway as the newest committer and PMC member of the > Apache Mesos project. > > Neil has been an active contributor to Mesos for more than a year now. As > part of his work, he has contributed som

Re: Welcome Guangya Liu as Mesos Committer and PMC member!

Welcome to the PMC! >>>>> > >>>>> >> On Fri, Dec 16, 2016 at 7:03 PM, Sam wrote: >>>>> >> congratulations Guangya >>>>> >> >>>>> >> Sent from my iPhone >>>>> >> >>>>> >

Re: Support for tasks groups aka pods in Mesos

The answer is No, the taskGroup can be treated as Pod in Kubernetes, it will be a set of containers co-located and co-managed on an agent that share some resources (e.g., network namespace, volumes). For your case, you may want to split your 50 tasks to small task groups. Also the `execute` cli is

Re: MESOS: Improve the performance of Resources.

Hi Joris, For `2x` number, can you please show more detail for how did you do and evaluate the test? How did you compare the result? Did you have any test code to share? Thanks, Guangya On Mon, Jul 11, 2016 at 6:05 PM, Klaus Ma wrote: > Hi Joris, > > For `Scalars`, yes, it's also dynamic allo

Re: MESOS-4694

Hi Ben and Dario, The reason that we have "SUPPRESS" call is as following: 1) Act as the complement to the current REVIVE call. 2) The HTTP API do not have an API to "Deactivate" a framework, we want to use "SUPPRESS", "DECLINE" and "DECLINE_INVERSE_OFFERS" to implement the call for "DeactivateFra

Allocator slack channel

Hi, I created an #allocator slack channel in mesos.slack.com, please join this if you want to have some discussion for allocator related issues, such as allocator performance, optimistic offer, revocable resource etc. @bmahler and @vinodkone, I also posted a RR here https://reviews.apache.org/r/

Re: Mesos CLI

t; > If we go for the Python route, I'd like to have a style guide for our > python code. Looks like we can directly use the google python style guide > <https://google.github.io/styleguide/pyguide.html>. Looks like pylint can > also check the style automatically. > > -

Re: Mesos CLI

+1 to use python. By using python, we can debug the CLI without re-compile but just update the CLI file and debug it with pdb, this is very helpful to trouble shooting. On Thu, Jun 23, 2016 at 9:34 AM, Kevin Klues wrote: > > > > The best option may still be for it > > to be in Python, this is wh

Re: [GPU] [Allocation] "Scarce" Resource Allocation

nt (i.e. we want to be able to schedule > with scarce resources successfully and we may have considerations like not > scheduling too many I/O bound workloads on a single host or more stringent > requirements for scheduling persistent tasks). > > > >  Elizabeth Lingg &

Re: [GPU] [Allocation] "Scarce" Resource Allocation

scarce resources before those scarce resource allocated to other frameworks. I'm planning to draft a document based on solution 1) "Create sub-pools" for the long term solution, any comments are welcome! Thanks, Guangya On Sat, Jun 18, 2016 at 11:58 AM, Guangya Liu wrote: > Tha

Re: [GPU] [Allocation] "Scarce" Resource Allocation

Thanks Du Fan. So you mean that we should have some clear rules in document or somewhere else to tell or guide cluster admin which resources should be classified as scarce resources, right? On Sat, Jun 18, 2016 at 2:38 AM, Du, Fan wrote: > > > On 2016/6/17 7:57, Guangya Liu wrote: >

Re: [GPU] [Allocation] "Scarce" Resource Allocation

Thanks all for the input here! @Hans van den Bogert, Yes, agree with Alex R, Mesos is now using coarse grained mode to allocate resources and the minimum unit is a single host, so you will always get cpu and memory. @Alex, Yes, I was only listing sorters here, ideally, I think that an indeal al

Re: [GPU] [Allocation] "Scarce" Resource Allocation

ere wrote: > With this 4th sorter approach, how does quota work for scarce resources? > > — > *Joris Van Remoortere* > Mesosphere > > On Thu, Jun 16, 2016 at 11:26 AM, Guangya Liu wrote: > > > Hi Ben, > > > > The pre-condition for four stage allocati

Re: [GPU] [Allocation] "Scarce" Resource Allocation

w adding another allocation stage for only scarce > resources would behave well? It seems to have a number of problems when I > think through it. > > On Sat, Jun 11, 2016 at 7:59 AM, Guangya Liu wrote: > >> Hi Ben, >> >> For long term goal, instead of creating s

Re: Apply for assignment of jira MESOS-5425

You may want to post an email to dev@mesos.apache.org to request add you as a contributor first. Please take a look at https://github.com/apache/mesos/blob/master/docs/newbie-guide.md#getting-started-guidance Thanks, Guangya On Sun, Jun 12, 2016 at 10:44 AM, Yan Yan YY Hu wrote: > > Dear Meso

Re: [GPU] [Allocation] "Scarce" Resource Allocation

Hi Ben, For long term goal, instead of creating sub-pool, what about adding a new sorter to handle **scare** resources? The current logic in allocator was divided to two stages: allocation for quota, allocation for non quota resources. I think that the future logic in allocator would be divided t

Re: Welcome Anand and Joseph as new committers!

Congrats to both, well deserved!! On Fri, Jun 10, 2016 at 5:00 AM, Vinod Kone wrote: > Hi folks, > > I'm happy to announce that the PMC has voted in *Anand Mazumdar *and > *Joseph > Wu* as committers and members of PMC for the Apache Mesos project. > > A little about the new committers. > > Anan

Re: [1/2] mesos git commit: Added aufs provisioning backend.

The document is being tracked here https://issues.apache.org/jira/browse/MESOS-5549 On Wed, Jun 8, 2016 at 7:29 PM, Neil Conway wrote: > Can you update the documentation for this change, please? > > Thanks, > Neil > > On Tue, Jun 7, 2016 at 6:14 PM, wrote: > > Repository: mesos > > Updated Bra

Re: how to debug HTTP API

e607ecc0f376ab62d60eb> > > -anand > > > On Jun 7, 2016, at 8:38 AM, Olivier Sallou > wrote: > > > > > > > > On 06/07/2016 04:53 PM, Guangya Liu wrote: > >> So how many agent nodes are there in your cluster? If you continue > >> re

Re: how to debug HTTP API

FFERS"? Thanks, Guangya On Tue, Jun 7, 2016 at 8:29 PM, Olivier Sallou wrote: > > > On 06/07/2016 01:59 PM, Guangya Liu wrote: > > I can see that your framework is now holding the offer, how did you > launch > > task? > > I execute an HTTP POST request in Py

Re: how to debug HTTP API

: > > > On 06/07/2016 12:25 PM, Guangya Liu wrote: > > Olivier, > > > > For such case, seems there is sth wrong with your framework? can you > please > > run the following two commands and check the output? > I don't think it is a framework issue, I recei

Re: how to debug HTTP API

Olivier, For such case, seems there is sth wrong with your framework? can you please run the following two commands and check the output? curl "http://:5050/master/frameworks" 2>/dev/null|python -m json.tool curl "http://:5050/master/state" 2>/dev/null|python -m json.tool Thanks, Guangya On Tu

Re: [MesosCon][Slides] Any slides sharing plan for MesosCon

sphere?iframe=yes&w=i:0;&sidebar=yes&bg=no > > On Fri, Jun 3, 2016 at 8:42 PM, Guangya Liu > wrote: > > > Unlike last year's MesosCon, I saw that the slides was not shared till > now > > in http://events.linuxfoundation.org/events/mesoscon-north-america , any > > plan to share those slides? > > > > Thanks, > > > > Guangya > > >

[MesosCon][Slides] Any slides sharing plan for MesosCon

Unlike last year's MesosCon, I saw that the slides was not shared till now in http://events.linuxfoundation.org/events/mesoscon-north-america , any plan to share those slides? Thanks, Guangya

Re: Documentation about debugging mesos-master : newbie

Hi Vinit, Please check if you are encountering this issue: https://github.com/Homebrew/homebrew-dupes/issues/221 Thanks, Guangya On Fri, Jun 3, 2016 at 2:24 AM, Vinit Mahedia wrote: > Hi Gilbert, > > Thank you for replying. > > Yes, I did that. > > >1. ./configure --enable-debug --disabl

Re: [VOTE] Release Apache Mesos 0.28.2 (rc1)

+1 to this. Only one minor comment: I saw that there are indeed someone using mesos CLI, such as `mesos ps` etc. I did minor enhancement for mesos CLI recently to improve debug-ability, it would be great if we can have those in 0.28.2. The following are the three patches for mesos CLI enhancement

Re: [Design Docs] A centralized place to store all design docs

It's here https://cwiki.apache.org/confluence/display/MESOS/Design+docs+--+Shared+Links On Thu, May 26, 2016 at 2:16 PM, Jay JN Guo wrote: > > Hi folk, > > Do we have a place (e.g. google driver directory) to aggregate all design > docs? I think it makes life easier for people to navigate and di

Re: Mesos Modules callbacks

Hi ct, There are not too much document for this, but you can refer to the following as reference and this may help. 1) https://github.com/apache/mesos/blob/master/docs/networking-for-mesos-managed-containers.md#writing-a-custom-network-isolator-module 2) https://github.com/apache/mesos/blob/maste

Re: volume / mount point error with Unified Containerizer

o track what's being fixed, working > on or planned in 0.29 and further releases? > > On Tue, May 24, 2016 at 1:51 AM, Guangya Liu wrote: > > > Olivier, > > > > If it is not urgent, I think that you can use 0.29 in next week, the > > unified container wil

Re: volume / mount point error with Unified Containerizer

-- > > De: "Guangya Liu" > > À: "dev" , "Jie Yu" , "Gilbert > Song" > > Envoyé: Lundi 23 Mai 2016 17:34:41 > > Objet: Re: volume / mount point error with Unified Containerizer > > > > It is a bit strange to me, I also did

Re: volume / mount point error with Unified Containerizer

nks, Guangya On Mon, May 23, 2016 at 10:30 PM, Guangya Liu wrote: > Thanks Olivier, I can reproduce this issue now and still checking what is > wrong. > > What I did is as following: > 1) Check out code with tag of 0.28.1 > 2) update mesos-execute to add a host path volume &

Re: volume / mount point error with Unified Containerizer

/runs/3239aafc-78d8-4f70-81e5-f32fb379/.rootfs/tmp/abcd does not exist Failed to execute a preparation shell command Will check more for this. Thanks, Guangya On Mon, May 23, 2016 at 3:35 PM, Olivier Sallou wrote: > > > On 05/23/2016 09:33 AM, Olivier Sallou wrote: > > >

Re: [REVIEW PROCESS] Proposal for new review process working group

+1, please count me in as well. Thanks, Guangya On Sat, May 21, 2016 at 2:11 AM, Shivam Pathak wrote: > Great! please add me to the group > > On Fri, May 20, 2016 at 11:07 AM, haosdent wrote: > > > This sounds great, add me to the group please. > > > > On Sat, May 21, 2016 at 1:59 AM, Kevin K

Re: volume / mount point error with Unified Containerizer

automatically created container mount points if they don't already > exist. > > > In the meantime, as a workaround for this, I've updated my filesystem > > > images to include the path (e.g. in Dockerfile, add `RUN mkdir -p > > > /some/mount/point`). Not the b

Re: volume / mount point error with Unified Containerizer

It's pretty simple for you from scratch with source code https://github.com/apache/mesos/blob/master/docs/getting-started.md#building-mesos ;-) Thanks, Guangya On Wed, May 18, 2016 at 8:30 PM, Olivier Sallou wrote: > > > On 05/18/2016 02:31 PM, Guangya Liu wrote: > > J

Re: volume / mount point error with Unified Containerizer

Just saw that you are working with 0.28.1, the "docker volume driver" code was not in 0.28.1, can you please have a try with mesos master branch if you are only doing some test? Thanks, Guangya On Wed, May 18, 2016 at 8:28 PM, Guangya Liu wrote: > Hi Olivier, > > I thi

Re: volume / mount point error with Unified Containerizer

Hi Olivier, I think that you need to enable "docker volume isolator" if you want use external storage with unified container I was writing a document here https://reviews.apache.org/r/47511/, perhaps you can have a try according to the document and post some comments there if you find any issues.

Re: Mesos admin REST API

n/latest/scheduler-http-api/ > > > > We don't do any authentication stuff in our Mesos system though. > > > > - Erik > > > > On Wed, May 18, 2016 at 1:47 AM, Guangya Liu wrote: > > > > > No, but there are some discussion and JIRA trac

Re: Mesos admin REST API

No, but there are some discussion and JIRA tracing this https://issues.apache.org/jira/browse/MESOS-3220 On Wed, May 18, 2016 at 4:08 PM, Olivier Sallou wrote: > Hi, > Is there any operator/admin admin to kill a task, via an admin API ? > > I faced issue where mesos does not send any offer

[Unified Container] [Unit Test] Pay attention to your test case when using "alpine" docker image with INTERNET filter

Hi, We got two JIRA tickets for same issue about test failure on CentOS 7: 1) https://issues.apache.org/jira/browse/MESOS-5351 2) https://issues.apache.org/jira/browse/MESOS-4810 The issue for those two JIRA tickets are the test case will be failed on CentOS 7, the root cause is that on some linu

[Allocator] Looking for a shepherd for MESOS-3765 (Make offer size adjustable)

Hi, Can anyone help to shepherd MESOS-3765 ? It would be a very important feature if customer running mesos on some powerful machines. Thanks, Guangya

Re: Design Doc for Qemu/KVM containerizer

@haosdent, You may see that I already linked MESOS-2717 to MESOS-3709 , yes, we will definitely consider the module mode. Thanks, Guangya On Thu, Apr 7, 2016 at 11:00 PM, haosdent wrote: > Hi, @Abhishek Thanks your nice design document. I ju

Re: [Mesos Agent][Isolator][Storage] Docker Volume Driver Isolator Design is ready for review

; > On Sun, Apr 3, 2016 at 9:45 PM, Guangya Liu wrote: > > > Hi, > > > > A design document for MESOS-4355 > > <https://issues.apache.org/jira/browse/MESOS-4355> > > > > > https://docs.google.com/document/d/1uhi1lf1_sEmnl0HaqHUCsqPb9m9jOKbRlXYW

[Mesos Agent][Isolator][Storage] Docker Volume Driver Isolator Design is ready for review

Hi, A design document for MESOS-4355 https://docs.google.com/document/d/1uhi1lf1_sEmnl0HaqHUCsqPb9m9jOKbRlXYW1S-tZis/edit?usp=sharing is ready for review, any comments are welcome. This project is enabling Mesos can integrate with external storag

Re: On launching command tasks

+1 on using docker mode, this can help the framework developer. Setting the command twice can sometimes make people confused. When I was working for the patch https://reviews.apache.org/r/1/ , I was also a bit confused before go through the code in agent part. On Sat, Apr 2, 2016 at 1:17 AM

Re: [Proposal] Use dev mailing list for working groups

+1. This is actually what OpenStack is doing now. OpenStack has many projects and every time we send an email to dev list, we always specify if it is for dev or user first, then specify the project as following: [openstack-dev][Magnum] Integrate Mesos Unified Container to Magnum. So for mesos, w

Re: Compile with CFLAGS=-DWITH_NETWORK_ISOLATOR

I did try this feature before, and you may want to follow here https://github.com/apache/mesos/blob/master/docs/network-monitoring.md#prerequisites to install the right version prerequisites first. On Tue, Mar 22, 2016 at 9:21 PM, Jay Guo wrote: > Hi, > > I got error trying to compile Mesos > on

Re: RFC: RevocableInfo Changes

Apologies if >>>> you guys already covered these questions in the meeting: >>>> >>>> If throttling is tolerable but preemption is not, how would that be >>>> expressed? (Is that supported?) >>>> >>>> How does this work with

Re: RFC: RevocableInfo Changes

Also please show your comments if any for the name here, the current name is *ThrottleInfo*, in Kubernetes resources qos design document, they are using scavenging as the key work for such behaviour, so a possible name here could be *ScavengeInfo , *please show your comments if any for those tw

Looking for shepherd (MESOS-4355 - Docker Volume Isolator)

Hi, I was now working on the FS for MESOS-4355 with some EMC guys, can anyone help shepherd for this? There are some issues need to discuss with the shepherd. Thanks, Guangya

Re: RFC: RevocableInfo Changes

resources type, two questions after the discussion: >> > > > >> > > > 1. should we send different offer to the framework, so when >> > > > usage/allocation oversubscription updated, only one type of offer >> will >> > be >> > &g

Re: RFC: RevocableInfo Changes

Hi Ben, I think that currently and even in the near future, the __ThrottleInfo__ will only be used by the usage oversubscriptions and the oversubscription for allocator (Both quota and reservations) will not use this value but only using __RevocableInfo__ is enough. I can even think that the

Re: [VOTE] Release Apache Mesos 0.28.0 (rc1)

There are also issues related with overlayfs, the overlayfs will not work if linux kernel is greater than 4.2, and also there is no document related to overlayfs. There are already patches and JIRA tickets for both issues. https://reviews.apache.org/r/44421/ https://reviews.apache.org/r/44391/ T

Re: 0.28.0 release

I think that we also need to include https://reviews.apache.org/r/44258/ ( https://issues.apache.org/jira/browse/MESOS-4831) in 0.28 cause this issue will lead two inverse offers with one maintain call. Thanks, Guangya On Fri, Mar 4, 2016 at 9:44 AM, Vinod Kone wrote: > Release vote sent. The

Re: Making 'curl' a prerequisite for installing Mesos

libcurl can automatically picks up certain environment variables and adjusts its settings accordingly, so libcurl support enabling http_proxy and https_proxy by default, this is important feature for someone who want to use a proxy to connect internet. One example is that I cannot get google docker

Re: Mesos supports to get the available total resource per-roles ?

Does /master/state endpoint help? This can get all resources per-role in the cluster, including total, used, reserved etc per role. *"reserved_resources"*: { *"r2"*: { *"mem"*: 0, *"disk"*: 0, *"cpus"*: 8 }, *"r1"*: { *"mem"*:

Re: Question about "Framework directly access Meso agent"

d to your comments and opinions. > > > > Best Regards, > > Teng > > > > > > > > Su Teng 00241668 > > > > Distributed and Parallel Software Lab > > Huawei Technologies Co., Ltd. > > Email:sut...@huawei.com > > > > > -- Guangya Liu (刘光亚) Senior Software Engineer DCOS and OpenStack Development IBM Platform Computing Systems and Technology Group

Re: Question about "Framework directly access Meso agent"

d to your comments and opinions. > > > > Best Regards, > > Teng > > > > > > > > Su Teng 00241668 > > > > Distributed and Parallel Software Lab > > Huawei Technologies Co., Ltd. > > Email:sut...@huawei.com > > > > > -- Guangya Liu (刘光亚) Senior Software Engineer DCOS and OpenStack Development IBM Platform Computing Systems and Technology Group

Re: [2/2] mesos git commit: Added documentation for labeled reserved resources.

Neil, I think what you want to show for setting reservationInfo for static reservation is https://issues.apache.org/jira/browse/MESOS-4476 On Sat, Feb 13, 2016 at 2:16 AM, Neil Conway wrote: > Hi Ben, > > On Fri, Feb 12, 2016 at 2:34 AM, Benjamin Mahler > wrote: > > Any plans to support labels

Re: Specifying a preferred host with a Resource Request

Hi Jagadish, Even though Mesos have the interface of "requestResources", it was not implemented in the built-in allocator at the moment, so the call of "driver.requestResources (resources);" will not work. Is it possible that you update your framework logic as this: 1) framework get resoruce offe

Re: work on Mesos Containerizer to support docker containers

I think that this is the entry point if you want to get to know more for latest container https://github.com/apache/mesos/blob/master/docs/mesos-provisioner.md On Tue, Jan 19, 2016 at 10:55 PM, Jan Schlicht wrote: > Hi Olivier, > > status for the "Unified Containerizer" project is tracked under

Re: 答复: Request Mesos contributor role

You can also take a look at following link to get more detail: https://github.com/apache/mesos/blob/master/docs/submitting-a-patch.md https://github.com/apache/mesos/blob/master/docs/effective-code-reviewing.md The last one is from OpenStack git commit, you can also take it as a reference: https:/

Re: Docker network support

There is a JIRA ticket here https://issues.apache.org/jira/browse/MESOS-3828 , you can append some of your requirement here to move this forward. Thanks! On Tue, Dec 8, 2015 at 8:41 PM, Olivier Sallou wrote: > Hi, > what is the current/planned feature support for Docker network ? > > Docker netw

Re: Docker network support

There is a JIRA ticket here https://issues.apache.org/jira/browse/MESOS-3828 , you can append some of your requirement here to move this forward. Thanks! On Tue, Dec 8, 2015 at 8:41 PM, Olivier Sallou wrote: > Hi, > what is the current/planned feature support for Docker network ? > > Docker netw

Re: Docker Executor in Mesos

Some comments in line. Thanks, Guangya On Tue, Dec 8, 2015 at 2:13 PM, Du, Fan wrote: > > So why not use one executor to launch docker tasks? >> > > Each task resides(or runs to be more precisely) in its own docker > container, > Every docker container is an executor by its nature. > If you la

Re: Configuration file?

+1000, introducing a new configuration file for mesos master and slave can help end user take the configuration file as the source of all flags. The OpenStack is also using same way to manage all of the flags, it is putting all flags into a configuration file and the configuration file including a

Mesos is helping OpenStack Container Deployment

Just FYI, the OpenStack Kolla Team [1] is forming a small team trying to investigate if Mesos could be used as an orchestration engine in place of Ansible [2], Mesos plus Marathon might be the solution. [1] https://github.com/openstack/kolla [2] http://markmail.org/thread/vg6xkavfpp6c4odr Thanks,

Re: [Breaking bug fix] Binary in state endpoints

+1 to remove the field directly, one comment is that the upgrade document may need to be updated. >From my understanding, since the data is binary data and I did not see too much requirement on retrieving binary data. Thanks! On Sat, Oct 24, 2015 at 5:33 AM, Joseph Wu wrote: > Hello, > > The s

Re: Community Sync Interval

+1 for bi-weekly Thanks, Guangya On Fri, Oct 16, 2015 at 6:08 AM, Daniel Mercer wrote: > +1 for weekly -- if this results in diminishing returns we can always reset > to biweekly. > > On Thu, Oct 15, 2015 at 2:44 PM, Kapil Arya wrote: > > > +1 for bi-weekly. > > > > On Thu, Oct 15, 2015 at 4:

Re: Do we still need to add InverseOffer support to Scheduler API?

E-mail: zhang...@cn.ibm.com <mailto:zhang...@cn.ibm.com> > > Chat: zhq527725 > > “An educated man should know everything about something and something > about everything" > > > > > > > > 陕西省西安市高新区 > > 高新六路42号中清大厦3层 > > Xian, Shaan

Re: Do we still need to add InverseOffer support to Scheduler API?

t? Thanks, Guangya On Mon, Sep 14, 2015 at 11:15 PM, haosdent wrote: > Hi @Guangya Liu. V1 API support both mesos call frameworks or frameworks > call mesos. > > > https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit > > And I think Java or

Re: Do we still need to add InverseOffer support to Scheduler API?

Hi Joris, I think that those APIs are still needed as HTTP API is mainly initiated by operator, the current call for HTTP API including TEARDOWN, ACCEPT, DECLINE, REVIVE, KILL, SHUTDOWN etc, but the offer related operations such as offer and InverserOffers are initiatedby mesos master, the master

Re: [Mesos][Patch] How to make the patches under review board link to a mesos bug

d the review request yourself. > > Kapil > > PS: The dev list doesn't allow images, so if you had attached any, they > were discarded :-). > > On Wed, Aug 5, 2015 at 5:52 PM, Guangya Liu wrote: > > > Hi Mesos, > > > > I see that there are some patches can b

[Mesos][Patch] How to make the patches under review board link to a mesos bug

Hi Mesos, I see that there are some patches can be easily be directed to a mesos bug by a link on the right side of the review board, how can I have this for my patch? ​ Thanks, Guangya

[mesos][gdb] How to use gdb-mesos-master.sh to debug mesos master code?

Hi Mesos dev: I'm now trying to use gdb-mesos-master.sh to debug mesos master but found that I cannot hit my break point, can someone help to see if there are something wrong with my steps? The following are my steps: *1) Start gdb-mesos-master.sh* root@devstack007:/home/gyliu/src/mesos/m1/mesos