Re: Altering agent resrouces after startup

2016-04-20 Thread haosdent 
>however I can't actually access that ticket.

Have you login in https://issues.apache.org/jira/browse/MESOS? Can you add
comments in it successfully?

On Wed, Apr 20, 2016 at 5:30 PM, Klaus Ma <klaus1982...@gmail.com> wrote:

> @Aaron, thanks for your info; I think MESOS-3059 cover your cases.
>
> On Wed, Apr 20, 2016 at 5:24 PM Aaron Carey <aca...@ilm.com> wrote:
>
>> 2 frameworks minimum, sometimes more (depends what we're doing at the
>> time). Marathon is always running and we have a couple of custom frameworks
>> too..
>>
>>
>> --
>>
>> Aaron Carey
>> Production Engineer - Cloud Pipeline
>> Industrial Light & Magic
>> London
>> 020 3751 9150
>>
>> --
>> *From:* Klaus Ma [klaus1982...@gmail.com]
>> *Sent:* 20 April 2016 10:22
>> *To:* user@mesos.apache.org
>>
>> *Subject:* Re: Altering agent resrouces after startup
>> For the label/attributes, I think this a case we need to pay attention:
>> the allocator did not count in label/attributes when doing allocation, so
>> the resources maybe assigned to different frameworks. @Aaron, how many
>> frameworks are you running?
>>
>> Thanks
>> Klaus
>>
>> On Wed, Apr 20, 2016 at 5:18 PM Aaron Carey <aca...@ilm.com> wrote:
>>
>>> Ah thank you! I tried searching Jira but didn't find that ticket.
>>>
>>> Yes I think you might be right about the attributes, although I don't
>>> seem to be able to get to the MESOS-3059 ticket in Jira, do you know if
>>> it's on the roadmap?
>>>
>>> Thanks,
>>> Aaron
>>>
>>> --
>>>
>>>
>>> Aaron Carey
>>> Production Engineer - Cloud Pipeline
>>> Industrial Light & Magic
>>> London
>>> 020 3751 9150
>>>
>>> --
>>> *From:* haosdent [haosd...@gmail.com]
>>> *Sent:* 20 April 2016 10:12
>>> *To:* user
>>> *Subject:* Re: Altering agent resrouces after startup
>>>
>>> There is a ticket [Allow slave reconfiguration on restart](
>>> https://issues.apache.org/jira/browse/MESOS-1739) related to this and
>>> not implement yet. But your requirement seems not related to change
>>> resources of agent dynamically. It looks more like change labels/attributes
>>> dynamically of agent.
>>>
>>> On Wed, Apr 20, 2016 at 4:56 PM, Aaron Carey <aca...@ilm.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I was wondering if it was possible somehow to alter an agent's
>>>> resources after it has started?
>>>>
>>>> Example: we are dynamically attaching and detaching EBS volumes to EC2
>>>> hosts running as agents. (This is part of our docker volume setup using
>>>> RexRay). When a host has an EBS volume attached to it I'd like to be able
>>>> to mark that as a new resource on the agent. Note that it's not the disk
>>>> space we care about here, just the name of the volume itself. This would
>>>> then allow us to schedule tasks that require access to the data on that EBS
>>>> volume all on the same host.
>>>>
>>>> Anyone have any ideas?
>>>>
>>>> Thanks!
>>>>
>>>> Aaron
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>> --
>>
>> Regards,
>> 
>> Da (Klaus), Ma (马达), PMP® | Advisory Software Engineer
>> IBM Platform Development & Support, STG, IBM GCG
>> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
>>
> --
>
> Regards,
> 
> Da (Klaus), Ma (马达), PMP® | Advisory Software Engineer
> IBM Platform Development & Support, STG, IBM GCG
> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
>



-- 
Best Regards,
Haosdent Huang

Re: Altering agent resrouces after startup

2016-04-20 Thread Klaus Ma 
@Aaron, thanks for your info; I think MESOS-3059 cover your cases.

On Wed, Apr 20, 2016 at 5:24 PM Aaron Carey <aca...@ilm.com> wrote:

> 2 frameworks minimum, sometimes more (depends what we're doing at the
> time). Marathon is always running and we have a couple of custom frameworks
> too..
>
>
> --
>
> Aaron Carey
> Production Engineer - Cloud Pipeline
> Industrial Light & Magic
> London
> 020 3751 9150
>
> --
> *From:* Klaus Ma [klaus1982...@gmail.com]
> *Sent:* 20 April 2016 10:22
> *To:* user@mesos.apache.org
>
> *Subject:* Re: Altering agent resrouces after startup
> For the label/attributes, I think this a case we need to pay attention:
> the allocator did not count in label/attributes when doing allocation, so
> the resources maybe assigned to different frameworks. @Aaron, how many
> frameworks are you running?
>
> Thanks
> Klaus
>
> On Wed, Apr 20, 2016 at 5:18 PM Aaron Carey <aca...@ilm.com> wrote:
>
>> Ah thank you! I tried searching Jira but didn't find that ticket.
>>
>> Yes I think you might be right about the attributes, although I don't
>> seem to be able to get to the MESOS-3059 ticket in Jira, do you know if
>> it's on the roadmap?
>>
>> Thanks,
>> Aaron
>>
>> --
>>
>>
>> Aaron Carey
>> Production Engineer - Cloud Pipeline
>> Industrial Light & Magic
>> London
>> 020 3751 9150
>>
>> --
>> *From:* haosdent [haosd...@gmail.com]
>> *Sent:* 20 April 2016 10:12
>> *To:* user
>> *Subject:* Re: Altering agent resrouces after startup
>>
>> There is a ticket [Allow slave reconfiguration on restart](
>> https://issues.apache.org/jira/browse/MESOS-1739) related to this and
>> not implement yet. But your requirement seems not related to change
>> resources of agent dynamically. It looks more like change labels/attributes
>> dynamically of agent.
>>
>> On Wed, Apr 20, 2016 at 4:56 PM, Aaron Carey <aca...@ilm.com> wrote:
>>
>>> Hi All,
>>>
>>> I was wondering if it was possible somehow to alter an agent's resources
>>> after it has started?
>>>
>>> Example: we are dynamically attaching and detaching EBS volumes to EC2
>>> hosts running as agents. (This is part of our docker volume setup using
>>> RexRay). When a host has an EBS volume attached to it I'd like to be able
>>> to mark that as a new resource on the agent. Note that it's not the disk
>>> space we care about here, just the name of the volume itself. This would
>>> then allow us to schedule tasks that require access to the data on that EBS
>>> volume all on the same host.
>>>
>>> Anyone have any ideas?
>>>
>>> Thanks!
>>>
>>> Aaron
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
> --
>
> Regards,
> 
> Da (Klaus), Ma (马达), PMP® | Advisory Software Engineer
> IBM Platform Development & Support, STG, IBM GCG
> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
>
-- 

Regards,

Da (Klaus), Ma (马达), PMP® | Advisory Software Engineer
IBM Platform Development & Support, STG, IBM GCG
+86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me

RE: Altering agent resrouces after startup

2016-04-20 Thread Aaron Carey 
Thanks Klaus,

After reading Haosdent's response I have a feeling a ticket may already exist: 
MESOS-3059 which would work for our usecase, however I can't actually access 
that ticket.

I'm happy to create a new one if needed though?

Thanks,
Aaron
--

Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150


From: Klaus Ma [klaus1982...@gmail.com]
Sent: 20 April 2016 10:18
To: user@mesos.apache.org
Subject: Re: Altering agent resrouces after startup

Hi Aaron,

Currently, the resources in slave can NOT be updated after started; the QoS can 
only report revocable resources. But I think this reasonable requirement to 
detect resource on the fly; would you help to open an JIRA for this? I think 
there're two sub-requirement of this scenario:

1. The resources of slave will be updated on the fly; it's different with 
MESOS-1739 which focus on agent restart
2. Self-defined resources which is only consumed special resources

If any comments, please let me know.

Thanks
Klaus


On Wed, Apr 20, 2016 at 4:56 PM Aaron Carey 
<aca...@ilm.com<mailto:aca...@ilm.com>> wrote:
Hi All,

I was wondering if it was possible somehow to alter an agent's resources after 
it has started?

Example: we are dynamically attaching and detaching EBS volumes to EC2 hosts 
running as agents. (This is part of our docker volume setup using RexRay). When 
a host has an EBS volume attached to it I'd like to be able to mark that as a 
new resource on the agent. Note that it's not the disk space we care about 
here, just the name of the volume itself. This would then allow us to schedule 
tasks that require access to the data on that EBS volume all on the same host.

Anyone have any ideas?

Thanks!


Aaron
--

Regards,

Da (Klaus), Ma (马达), PMP® | Advisory Software Engineer
IBM Platform Development & Support, STG, IBM GCG
+86-10-8245 4084 | mad...@cn.ibm.com<mailto:mad...@cn.ibm.com> | http://k82.me

RE: Altering agent resrouces after startup

2016-04-20 Thread Aaron Carey 
Ah thank you! I tried searching Jira but didn't find that ticket.

Yes I think you might be right about the attributes, although I don't seem to 
be able to get to the MESOS-3059 ticket in Jira, do you know if it's on the 
roadmap?

Thanks,
Aaron

--

Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150


From: haosdent [haosd...@gmail.com]
Sent: 20 April 2016 10:12
To: user
Subject: Re: Altering agent resrouces after startup

There is a ticket [Allow slave reconfiguration on 
restart](https://issues.apache.org/jira/browse/MESOS-1739) related to this and 
not implement yet. But your requirement seems not related to change resources 
of agent dynamically. It looks more like change labels/attributes dynamically 
of agent.

On Wed, Apr 20, 2016 at 4:56 PM, Aaron Carey 
<aca...@ilm.com<mailto:aca...@ilm.com>> wrote:
Hi All,

I was wondering if it was possible somehow to alter an agent's resources after 
it has started?

Example: we are dynamically attaching and detaching EBS volumes to EC2 hosts 
running as agents. (This is part of our docker volume setup using RexRay). When 
a host has an EBS volume attached to it I'd like to be able to mark that as a 
new resource on the agent. Note that it's not the disk space we care about 
here, just the name of the volume itself. This would then allow us to schedule 
tasks that require access to the data on that EBS volume all on the same host.

Anyone have any ideas?

Thanks!

Aaron



--
Best Regards,
Haosdent Huang

Re: Altering agent resrouces after startup

2016-04-20 Thread Klaus Ma 
Hi Aaron,

Currently, the resources in slave can NOT be updated after started; the QoS
can only report revocable resources. But I think this reasonable
requirement to detect resource on the fly; would you help to open an JIRA
for this? I think there're two sub-requirement of this scenario:

1. The resources of slave will be updated on the fly; it's different with
MESOS-1739 which focus on agent restart
2. Self-defined resources which is only consumed special resources

If any comments, please let me know.

Thanks
Klaus


On Wed, Apr 20, 2016 at 4:56 PM Aaron Carey  wrote:

> Hi All,
>
> I was wondering if it was possible somehow to alter an agent's resources
> after it has started?
>
> Example: we are dynamically attaching and detaching EBS volumes to EC2
> hosts running as agents. (This is part of our docker volume setup using
> RexRay). When a host has an EBS volume attached to it I'd like to be able
> to mark that as a new resource on the agent. Note that it's not the disk
> space we care about here, just the name of the volume itself. This would
> then allow us to schedule tasks that require access to the data on that EBS
> volume all on the same host.
>
> Anyone have any ideas?
>
> Thanks!
>
>
> Aaron
>
-- 

Regards,

Da (Klaus), Ma (马达), PMP® | Advisory Software Engineer
IBM Platform Development & Support, STG, IBM GCG
+86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me

Re: Altering agent resrouces after startup

2016-04-20 Thread haosdent 
There is a ticket [Allow slave reconfiguration on restart](
https://issues.apache.org/jira/browse/MESOS-1739) related to this and not
implement yet. But your requirement seems not related to change resources
of agent dynamically. It looks more like change labels/attributes
dynamically of agent.

On Wed, Apr 20, 2016 at 4:56 PM, Aaron Carey  wrote:

> Hi All,
>
> I was wondering if it was possible somehow to alter an agent's resources
> after it has started?
>
> Example: we are dynamically attaching and detaching EBS volumes to EC2
> hosts running as agents. (This is part of our docker volume setup using
> RexRay). When a host has an EBS volume attached to it I'd like to be able
> to mark that as a new resource on the agent. Note that it's not the disk
> space we care about here, just the name of the volume itself. This would
> then allow us to schedule tasks that require access to the data on that EBS
> volume all on the same host.
>
> Anyone have any ideas?
>
> Thanks!
>
> Aaron
>



-- 
Best Regards,
Haosdent Huang

Altering agent resrouces after startup

2016-04-20 Thread Aaron Carey 
Hi All,

I was wondering if it was possible somehow to alter an agent's resources after 
it has started?

Example: we are dynamically attaching and detaching EBS volumes to EC2 hosts 
running as agents. (This is part of our docker volume setup using RexRay). When 
a host has an EBS volume attached to it I'd like to be able to mark that as a 
new resource on the agent. Note that it's not the disk space we care about 
here, just the name of the volume itself. This would then allow us to schedule 
tasks that require access to the data on that EBS volume all on the same host.

Anyone have any ideas?

Thanks!

Aaron