from:"Benson Qiu \(JIRA\)"

[jira] [Commented] (YARN-6164) Expose Queue Configurations per Node Label through YARN client api

2017-04-24 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981416#comment-15981416
 ] 

Benson Qiu commented on YARN-6164:
--

Thanks [~sunilg] for committing this patch! Thanks [~bibinchundatt], 
[~leftnoteasy], [~rohithsharma], [~churromorales]!

> Expose Queue Configurations per Node Label through YARN client api
> --
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch, 
> YARN-6164-branch-2.001.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose Queue Configurations per Node Label through YARN client api

2017-04-20 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977105#comment-15977105
 ] 

Benson Qiu commented on YARN-6164:
--

Looks like the build is fixed now (YARN-6501).

> Expose Queue Configurations per Node Label through YARN client api
> --
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch, 
> YARN-6164-branch-2.001.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose Queue Configurations per Node Label through YARN client api

2017-04-20 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: (was: YARN-6164.009.patch)

> Expose Queue Configurations per Node Label through YARN client api
> --
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch, 
> YARN-6164-branch-2.001.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose Queue Configurations per Node Label through YARN client api

2017-04-19 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164-branch-2.001.patch

> Expose Queue Configurations per Node Label through YARN client api
> --
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch, 
> YARN-6164.009.patch, YARN-6164-branch-2.001.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose Queue Configurations per Node Label through YARN client api

2017-04-19 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.009.patch

> Expose Queue Configurations per Node Label through YARN client api
> --
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch, 
> YARN-6164.009.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-04-16 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15970644#comment-15970644
 ] 

Benson Qiu commented on YARN-6164:
--

[~sunilg]: Is it okay to move forward with this patch?

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-11 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964500#comment-15964500
 ] 

Benson Qiu edited comment on YARN-6195 at 4/11/17 3:17 PM:
---

Thanks so much [~jlowe] for all of your help throughout this process, and 
thanks [~Naganarasimha], [~leftnoteasy], and [~sunilg] for taking the time to 
review my patch as well! I really appreciate it!


was (Author: benson.qiu):
Thanks so much [~jlowe] for all of your help throughout this process, and 
thanks Naga, Wangda, and Sunil for taking the time to review my patch as well! 
I really appreciate it!

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-11 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964500#comment-15964500
 ] 

Benson Qiu commented on YARN-6195:
--

Thanks so much [~jlowe] for all of your help throughout this process, and 
thanks Naga, Wangda, and Sunil for taking the time to review my patch as well! 
I really appreciate it!

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-10 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963674#comment-15963674
 ] 

Benson Qiu commented on YARN-6195:
--

Thanks Jason and Naga for taking a look at my latest patch!

[~jlowe]: Do you happen to have the answer to Naga's question about making 
modifications to both hadoop-yarn-project and hadoop-common-project?

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-04-07 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15961450#comment-15961450
 ] 

Benson Qiu commented on YARN-6164:
--

[~sunilg]: Mind taking a look at the latest patch?

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-05 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958090#comment-15958090
 ] 

Benson Qiu commented on YARN-6195:
--

Version 5 outputs usedCapacity and absoluteUsedCapacity for the default 
partition.

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-05 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6195:
-
Attachment: YARN-6195.005.patch

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-05 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957532#comment-15957532
 ] 

Benson Qiu commented on YARN-6195:
--

I will fix the findbugs issue from HadoopQA.

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch, YARN-6195.004.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-05 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6195:
-
Attachment: YARN-6195.004.patch

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch, YARN-6195.004.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-05 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956608#comment-15956608
 ] 

Benson Qiu edited comment on YARN-6195 at 4/5/17 10:01 AM:
---

Thanks [~Naganarasimha] and [~leftnoteasy] for the comments. Your explanations 
for why you prefer the default partition over the max across all partitions 
makes sense. The max across all partitions might be less useful since we don't 
have a way of reporting the label associated with the max value.

[~jlowe]: What are your thoughts on this?


was (Author: benson.qiu):
Thanks [~Naganarasimha] and [~leftnoteasy] for the comments. Your explanations 
for why you prefer the default partition over the max across all partitions 
makes sense. The max across all partitions might be less useful since we don't 
have a way of reporting the label associated with the max value.

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-05 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956608#comment-15956608
 ] 

Benson Qiu commented on YARN-6195:
--

Thanks [~Naganarasimha] and [~leftnoteasy] for the comments. Your explanations 
for why you prefer the default partition over the max across all partitions 
makes sense. The max across all partitions might be less useful since we don't 
have a way of reporting the label associated with the max value.

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-04-04 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955497#comment-15955497
 ] 

Benson Qiu commented on YARN-6195:
--

[~jlowe]: Thanks for your most recent review. If nobody has any objections, is 
it okay to proceed with this patch?

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-03-30 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15949823#comment-15949823
 ] 

Benson Qiu commented on YARN-6164:
--

Thanks [~sunilg] for the review.

Here are my the changes for version 8 based on your comments:

# Added Apache license header to QueueConfigurations and 
QueueConfigurationsPBImpl.
# I ran TestRMRestart.testFinishedAppRemovalAfterRMRestart several times 
locally, seems flaky.
# Fixed most of the checkstyle comments.
# Made QueueConfigurationsProto optional in yarn_protos.proto
# Uncommented setCapacity in QueueConfigurationPBImpl.java. Thanks for catching 
that.


> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-03-30 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.008.patch

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch, YARN-6164.008.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-03-22 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937501#comment-15937501
 ] 

Benson Qiu commented on YARN-6164:
--

[~sunilg]: Version 7 contains the changes you mentioned yesterday. 

TestRMAdminService passes for me locally.

I will fix the whitespace error in the next patch.

Please let me know if there are any other changes I should make.

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-03-22 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.007.patch

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch, YARN-6164.007.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-03-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6195:
-
Attachment: YARN-6195.003.patch

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch, 
> YARN-6195.003.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-03-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.006.patch

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch, 
> YARN-6164.006.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-03-13 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922897#comment-15922897
 ] 

Benson Qiu edited comment on YARN-6195 at 3/13/17 8:45 PM:
---

[~leftnoteasy], [~sunilg]: Here's the approach I have right now, based on 
[~jlowe]'s comments. Would appreciate any thoughts.

{code:title=CSQueueUtils#updateUsedCapacity|borderStyle=solid}
Set nodeLabels = childQueue.getNodeLabelsForQueue();

// {some code omitted}

// QueueMetrics does not support per-label capacities,
// so we report the maximum across all labels.
for (String nodeLabel : nodeLabels) {
  usedCapacity =
  Math.max(queueCapacities.getUsedCapacity(nodeLabel), usedCapacity);
  absoluteUsedCapacity =
  Math.max(queueCapacities.getAbsoluteUsedCapacity(nodeLabel),
  absoluteUsedCapacity);
}
queueMetrics.setUsedCapacity(usedCapacity);
queueMetrics.setAbsoluteUsedCapacity(absoluteUsedCapacity);
{code}


was (Author: benson.qiu):
[~leftnoteasy], [~sunilg]]: Here's the approach I have right now, based on 
[~jlowe]'s comments. Would appreciate any thoughts.

{code:title=CSQueueUtils#updateUsedCapacity|borderStyle=solid}
Set nodeLabels = childQueue.getNodeLabelsForQueue();

// {some code omitted}

// QueueMetrics does not support per-label capacities,
// so we report the maximum across all labels.
for (String nodeLabel : nodeLabels) {
  usedCapacity =
  Math.max(queueCapacities.getUsedCapacity(nodeLabel), usedCapacity);
  absoluteUsedCapacity =
  Math.max(queueCapacities.getAbsoluteUsedCapacity(nodeLabel),
  absoluteUsedCapacity);
}
queueMetrics.setUsedCapacity(usedCapacity);
queueMetrics.setAbsoluteUsedCapacity(absoluteUsedCapacity);
{code}

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-03-13 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922897#comment-15922897
 ] 

Benson Qiu commented on YARN-6195:
--

[~leftnoteasy], [~sunilg]]: Here's the approach I have right now, based on 
[~jlowe]'s comments. Would appreciate any thoughts.

{code:title=CSQueueUtils#updateUsedCapacity|borderStyle=solid}
Set nodeLabels = childQueue.getNodeLabelsForQueue();

// {some code omitted}

// QueueMetrics does not support per-label capacities,
// so we report the maximum across all labels.
for (String nodeLabel : nodeLabels) {
  usedCapacity =
  Math.max(queueCapacities.getUsedCapacity(nodeLabel), usedCapacity);
  absoluteUsedCapacity =
  Math.max(queueCapacities.getAbsoluteUsedCapacity(nodeLabel),
  absoluteUsedCapacity);
}
queueMetrics.setUsedCapacity(usedCapacity);
queueMetrics.setAbsoluteUsedCapacity(absoluteUsedCapacity);
{code}

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-03-13 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6195:
-
Attachment: YARN-6195.002.patch

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch, YARN-6195.002.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-03-09 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902768#comment-15902768
 ] 

Benson Qiu commented on YARN-6195:
--

Thanks for the comments [~jlowe]!

The current JMX QueueMetrics don't seem to take into account node labels - can 
we do the same for usedCapacity and absoluteUsedCapacity?

The {{usedCapacity}} case would look like this:

{code:title=AbstractCSQueue.java|borderStyle=solid}
  @Override
  public void setUsedCapacity(float usedCapacity) {
queueCapacities.setUsedCapacity(usedCapacity);
metrics.setUsedCapacity(usedCapacity);
  }
  
  @Override
  public void setUsedCapacity(String label, float usedCapacity) {
queueCapacities.setUsedCapacity(label, usedCapacity);
if (label == null || label == CommonNodeLabelsManager.NO_LABEL) {
  metrics.setUsedCapacity(usedCapacity);
}
  }
{code}

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: capacityscheduler, metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-03-08 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901696#comment-15901696
 ] 

Benson Qiu commented on YARN-6164:
--

[~sunilg], [~leftnoteasy]: Sure, I can add {{QueueCapacities}} to {{QueueInfo}} 
as originally discussed.

If I work on this, is it very likely that my patch will get accepted? I would 
be working on this patch on my own time (not for the company I work for).

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-03-06 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897685#comment-15897685
 ] 

Benson Qiu commented on YARN-6164:
--

[~sunilg] [~bibinchundatt]

I'm planning to set aside some time this week to work on creating 
`QueueConfigurations`.

I would like to apply this patch to 2.8.0 as well.

If I complete the feature discussed above (add a per-label map of 
{{QueueConfigurations}}, what is the likelihood that my patch will be accepted?


> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-02-27 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886150#comment-15886150
 ] 

Benson Qiu commented on YARN-6164:
--

Thanks for the comments [~sunilg].

So 
[QueueInfo|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/api/records/QueueInfo.html]
 would expose something like `Map 
getQueueConfigurations()`.

`QueueInfo` already has `getCapacity()` and `getMaximumCapacity()`, which 
currently doesn't take into account node-labels. Are there any concerns with 
breaking backwards compatibility if we delete these two methods from the 
interface?

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-02-21 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.005.patch

005: Fix org.apache.hadoop.yarn.api.TestPBImplRecords.testQueueInfoPBImpl

The other failing tests from version 004 pass on my local machine.

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch, YARN-6164.005.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-02-19 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873959#comment-15873959
 ] 

Benson Qiu commented on YARN-6164:
--

[~sunilg]: I added per-label-per-queue AM resource percentages.

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-02-19 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873957#comment-15873957
 ] 

Benson Qiu edited comment on YARN-6164 at 2/20/17 2:22 AM:
---

Version 004: per-node-label max AM limits.


was (Author: benson.qiu):
004: per-node-label max AM limits.

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-02-19 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.004.patch

004: per-node-label max AM limits.

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch, YARN-6164.004.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-02-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6195:
-
Flags:   (was: Patch)

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-02-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6195:
-
Flags: Patch

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-02-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6195:
-
Attachment: YARN-6195.001.patch

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6195.001.patch
>
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-02-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu reassigned YARN-6164:


Assignee: Benson Qiu  (was: Benson Qiu)

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>Assignee: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-02-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu reassigned YARN-6195:


Assignee: Benson Qiu  (was: Benson Qiu)

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-02-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu reassigned YARN-6195:


Assignee: Benson Qiu

> Export UsedCapacity and AbsoluteUsedCapacity to JMX
> ---
>
> Key: YARN-6195
> URL: https://issues.apache.org/jira/browse/YARN-6195
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: metrics, yarn
>Affects Versions: 3.0.0-alpha3
>Reporter: Benson Qiu
>Assignee: Benson Qiu
>
> `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Created] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX

2017-02-15 Thread Benson Qiu (JIRA)

Benson Qiu created YARN-6195:


 Summary: Export UsedCapacity and AbsoluteUsedCapacity to JMX
 Key: YARN-6195
 URL: https://issues.apache.org/jira/browse/YARN-6195
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: metrics, yarn
Affects Versions: 3.0.0-alpha3
Reporter: Benson Qiu


`usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent in YarnClient

2017-02-15 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Description: 
`yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
[Cluster Scheduler 
API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
 but not through 
[YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].

Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
default), it would be nice to expose `maximum-am-resource-percent` in 
YarnClient as well. 

  was:
To my knowledge, there is no way to programmatically obtain 
`yarn.scheduler.capacity.maximum-am-resource-percent`.

I've tried looking at 
[YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
 [ResourceManager REST 
APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
 and 
[YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
 (via bin/yarn) 

Summary: Expose maximum-am-resource-percent in YarnClient  (was: Expose 
maximum-am-resource-percent to Hadoop clients)

> Expose maximum-am-resource-percent in YarnClient
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch
>
>
> `yarn.scheduler.capacity.maximum-am-resource-percent` is exposed through the 
> [Cluster Scheduler 
> API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API],
>  but not through 
> [YarnClient|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/client/api/YarnClient.html].
> Since YarnClient and RM REST APIs depend on different ports (8032 vs 8088 by 
> default), it would be nice to expose `maximum-am-resource-percent` in 
> YarnClient as well. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-14 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866995#comment-15866995
 ] 

Benson Qiu commented on YARN-6164:
--

Version 003: Fix test failures

> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch
>
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-am-resource-percent`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-14 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.003.patch

> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch, 
> YARN-6164.003.patch
>
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-am-resource-percent`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-13 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.002.patch

Version 002: Overloaded QueueInfo.newInstance instead of modifying the method 
signature. Thanks [~churromorales] for the suggestion.

> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
> Attachments: YARN-6164.001.patch, YARN-6164.002.patch
>
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-am-resource-percent`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-09 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Description: 
To my knowledge, there is no way to programmatically obtain 
`yarn.scheduler.capacity.maximum-am-resource-percent`.

I've tried looking at 
[YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
 [ResourceManager REST 
APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
 and 
[YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
 (via bin/yarn) 

  was:
To my knowledge, there is no way to programmatically obtain 
`yarn.scheduler.capacity.maximum-applications`.

I've tried looking at 
[YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
 [ResourceManager REST 
APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
 and 
[YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
 (via bin/yarn) 


> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
> Attachments: YARN-6164.001.patch
>
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-am-resource-percent`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-09 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Target Version/s: 3.0.0-alpha3

> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
> Attachments: YARN-6164.001.patch
>
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-applications`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-09 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Attachment: YARN-6164.001.patch

> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
> Attachments: YARN-6164.001.patch
>
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-applications`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-09 Thread Benson Qiu (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benson Qiu updated YARN-6164:
-
Description: 
To my knowledge, there is no way to programmatically obtain 
`yarn.scheduler.capacity.maximum-applications`.

I've tried looking at 
[YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
 [ResourceManager REST 
APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
 and 
[YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
 (via bin/yarn) 

  was:
To my knowledge, there is no way to programmatically obtain 
`yarn.scheduler.capacity.maximum-applications`.

I've tried looking at 
[YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
 [ResourceManager REST 
APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
 and 
[YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
 (via bin/yarn.sh) 


> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-applications`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-09 Thread Benson Qiu (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860357#comment-15860357
 ] 

Benson Qiu commented on YARN-6164:
--

I built the latest version of Hadoop and it appears that 
[maxAMLimitPercentage|https://github.com/apache/hadoop/blob/f67237cbe7bc48a1b9088e990800b37529f1db2a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/QueueCapacitiesInfo.java#L52]
 has been added to the Cluster Scheduler API (/ws/v1/cluster/scheduler).

I still don't believe this config is exposed through YarnClient or YarnCLI 
though.

> Expose maximum-am-resource-percent to Hadoop clients
> 
>
> Key: YARN-6164
> URL: https://issues.apache.org/jira/browse/YARN-6164
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: Benson Qiu
>
> To my knowledge, there is no way to programmatically obtain 
> `yarn.scheduler.capacity.maximum-applications`.
> I've tried looking at 
> [YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
>  [ResourceManager REST 
> APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
>  and 
> [YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
>  (via bin/yarn.sh) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Created] (YARN-6164) Expose maximum-am-resource-percent to Hadoop clients

2017-02-08 Thread Benson Qiu (JIRA)

Benson Qiu created YARN-6164:


 Summary: Expose maximum-am-resource-percent to Hadoop clients
 Key: YARN-6164
 URL: https://issues.apache.org/jira/browse/YARN-6164
 Project: Hadoop YARN
  Issue Type: Improvement
Affects Versions: 2.7.2
Reporter: Benson Qiu


To my knowledge, there is no way to programmatically obtain 
`yarn.scheduler.capacity.maximum-applications`.

I've tried looking at 
[YarnClient|https://hadoop.apache.org/docs/r2.7.2/api/index.html?org/apache/hadoop/yarn/client/api/YarnClient.html],
 [ResourceManager REST 
APIs|https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html],
 and 
[YarnCommands|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YarnCommands.html]
 (via bin/yarn.sh) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

51 matches

Mail list logo