[jira] [Assigned] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild

2017-07-31 Thread Junping Du (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du reassigned MAPREDUCE-5875:
-

Assignee: Gera Shegalov  (was: Junping Du)

> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch, 
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch, 
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch, 
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by 
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized 
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas 
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max 
> counters/groups is only increased when needed. With the current Limits 
> implementation relying on static constants, it's going to be challenging for 
> tools that submit jobs concurrently  without resorting to class loading 
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Assigned] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild

2017-07-31 Thread Junping Du (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du reassigned MAPREDUCE-5875:
-

Assignee: Junping Du  (was: Gera Shegalov)

> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Junping Du
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch, 
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch, 
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch, 
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by 
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized 
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas 
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max 
> counters/groups is only increased when needed. With the current Limits 
> implementation relying on static constants, it's going to be challenging for 
> tools that submit jobs concurrently  without resorting to class loading 
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org