[jira] [Updated] (MAPREDUCE-3628) DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3628: - Priority: Blocker (was: Critical) > DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 > nodes size cluster. > - > > Key: MAPREDUCE-3628 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3628 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre >Priority: Blocker > Fix For: 0.23.1 > > > DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 > nodes size cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3631) Corner case in headroom calculation
[ https://issues.apache.org/jira/browse/MAPREDUCE-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy resolved MAPREDUCE-3631. -- Resolution: Duplicate Fixed via MAPREDUCE-3126. > Corner case in headroom calculation > --- > > Key: MAPREDUCE-3631 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3631 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre > Fix For: 0.23.1 > > > When there is a single queue and a large job fills up the whole cluster, a > lost NM can lead to wrong headroom when all > slots are taken up by reduces since at that point headroom isn't recomputed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3622) findbug error during test-patch: org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent defines equals but not hashCode
[ https://issues.apache.org/jira/browse/MAPREDUCE-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved MAPREDUCE-3622. Resolution: Duplicate > findbug error during test-patch: > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent > defines equals but not hashCode > --- > > Key: MAPREDUCE-3622 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3622 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves > > findbug error: > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent > defines equals but not hashCode > first time I see this is build 1540: > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1540//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3628) DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181113#comment-13181113 ] Todd Lipcon commented on MAPREDUCE-3628: Which test case are you using here? Is it TestDFSIO or something closer to a "teravalidate" type workload? Is the input dataset larger than RAM or are you disk-bound? > DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 > nodes size cluster. > - > > Key: MAPREDUCE-3628 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3628 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre >Priority: Critical > Fix For: 0.23.1 > > > DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 > nodes size cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3619) Change streaming code to use new mapreduce api.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181098#comment-13181098
]
Liyin Liang commented on MAPREDUCE-3619:
Sorry. MAPREDUCE-1813 has fixed the NPE problem. They met the same bug with us.
> Change streaming code to use new mapreduce api.
> ---
>
> Key: MAPREDUCE-3619
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3619
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/streaming, mrv2
>Affects Versions: 0.23.1
>Reporter: Liyin Liang
>
> If we run a streaming job with following python script as mapper or reducer,
> the job will throws NullPointerException.
> {code:}
> #!/usr/bin/python
> import sys,os
> class MyTask:
> def __init__(self, file=sys.stdin):
> self.file = file
> print >>sys.stderr, "reporter:counter:spam,disp_flag_record,0"
> print >>sys.stderr, "reporter:counter:spam,spam_record,0"
> def process(self):
> while True:
> line = self.file.readline()
> if not line:
> break;
> print line
> if __name__ == "__main__":
> task = MyTask()
> task.process()
> {code}
> Here is the NPE related log:
> 2011-12-22 14:14:06,310 WARN org.apache.hadoop.streaming.PipeMapRed:
> java.lang.NullPointerException
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.incrCounter(PipeMapRed.java:502)
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:444)
> This is because the above script's "print >>sys.stderr" will invoke
> reporter.incrCounter() during PipeMapper|PipeReducer.configure(). While we
> can not get reporter in configure() function.
> To fix this problem, we should change streaming code to use new-api. Then we
> can call context.getCounter() in Mapper|Reducer.setup() function.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3629) Remove sleep from MRAppMaster during app-finish.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3629: --- Priority: Major (was: Critical) That unnecessary sleep still exists but it isn't really a critical issue. I suspect it isn't straightforward too, have to be careful with tests etc. > Remove sleep from MRAppMaster during app-finish. > > > Key: MAPREDUCE-3629 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3629 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre > Fix For: 0.23.1 > > > MRAppMaster waits for 5 secs during app-finish, this was needed before we had > client-side redirection. This affects the app execution in that, AppMaster > will killed by the NM once NM gets > confirmation from RM. > AppMaster should go away immediately. Also, the done call to RM from AM > should be the last thing AM ever does. > Otherwise, today, JobHistory writing gets interrupted if AM gets killed by > the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3543) Mavenize Gridmix.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181094#comment-13181094 ] Amar Kamat commented on MAPREDUCE-3543: --- Alejandro, Rumen was designed to be a contract that MapReduce provides for tools and users depending on MapReduce JobHistory. Rumen should be packaged with Hadoop MapReduce. Rumen might need access to internal MapReduce classes. Currently, only Gridmix uses Rumen but in future Mumak will also be using it. > Mavenize Gridmix. > - > > Key: MAPREDUCE-3543 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3543 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Mahadev konar >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3543v1.patch, MAPREDUCE-3543v1.sh > > > Gridmix codebase still resides in src/contrib and needs to be compiled via > ant. We should move it to maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3632) Need better error message on the Web UI when NM can't find the container logs instead of NPEno
[ https://issues.apache.org/jira/browse/MAPREDUCE-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Gummadi updated MAPREDUCE-3632: Attachment: 3632.patch Attaching patch fixing NPE issue and causes resulting into better error message when container logs are not there. > Need better error message on the Web UI when NM can't find the container logs > instead of NPEno > -- > > Key: MAPREDUCE-3632 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3632 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.0 >Reporter: Ravi Gummadi >Assignee: Ravi Gummadi > Fix For: 0.23.1 > > Attachments: 3632.patch > > > If for some reason NM could not find container logs, then an NPE is seen > while trying to access from web UI. Instead an error message should be > displayed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3632) Need better error message on the Web UI when NM can't find the container logs instead of NPEno
[ https://issues.apache.org/jira/browse/MAPREDUCE-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181089#comment-13181089 ] Ravi Gummadi commented on MAPREDUCE-3632: - Here are the NPE details: java.lang.NullPointerException at org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage$ContainersLogsBlock.printLogs(ContainerLogsPage.java:278) at org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage$ContainersLogsBlock.render(ContainerLogsPage.java:164) at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:64) at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74) at org.apache.hadoop.yarn.webapp.View.render(View.java:233) at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:47) at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117) at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:843) at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:54) at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:80) at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:210) at org.apache.hadoop.yarn.server.nodemanager.webapp.NMController.logs(NMController.java:104) ... 33 more > Need better error message on the Web UI when NM can't find the container logs > instead of NPEno > -- > > Key: MAPREDUCE-3632 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3632 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.0 >Reporter: Ravi Gummadi >Assignee: Ravi Gummadi > Fix For: 0.23.1 > > > If for some reason NM could not find container logs, then an NPE is seen > while trying to access from web UI. Instead an error message should be > displayed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3632) Need better error message on the Web UI when NM can't find the container logs instead of NPEno
Need better error message on the Web UI when NM can't find the container logs instead of NPEno -- Key: MAPREDUCE-3632 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3632 Project: Hadoop Map/Reduce Issue Type: Bug Components: nodemanager Affects Versions: 0.23.0 Reporter: Ravi Gummadi Assignee: Ravi Gummadi Fix For: 0.23.1 If for some reason NM could not find container logs, then an NPE is seen while trying to access from web UI. Instead an error message should be displayed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3619) Change streaming code to use new mapreduce api.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181070#comment-13181070
]
Mahadev konar commented on MAPREDUCE-3619:
--
I dont think we have to make streaming work with the new API. We should just
get it working with the old API. Putting in a hack so that it works with the
old api's during init() should be fine. no?
> Change streaming code to use new mapreduce api.
> ---
>
> Key: MAPREDUCE-3619
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3619
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/streaming, mrv2
>Affects Versions: 0.23.1
>Reporter: Liyin Liang
>
> If we run a streaming job with following python script as mapper or reducer,
> the job will throws NullPointerException.
> {code:}
> #!/usr/bin/python
> import sys,os
> class MyTask:
> def __init__(self, file=sys.stdin):
> self.file = file
> print >>sys.stderr, "reporter:counter:spam,disp_flag_record,0"
> print >>sys.stderr, "reporter:counter:spam,spam_record,0"
> def process(self):
> while True:
> line = self.file.readline()
> if not line:
> break;
> print line
> if __name__ == "__main__":
> task = MyTask()
> task.process()
> {code}
> Here is the NPE related log:
> 2011-12-22 14:14:06,310 WARN org.apache.hadoop.streaming.PipeMapRed:
> java.lang.NullPointerException
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.incrCounter(PipeMapRed.java:502)
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:444)
> This is because the above script's "print >>sys.stderr" will invoke
> reporter.incrCounter() during PipeMapper|PipeReducer.configure(). While we
> can not get reporter in configure() function.
> To fix this problem, we should change streaming code to use new-api. Then we
> can call context.getCounter() in Mapper|Reducer.setup() function.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3619) Change streaming code to use new mapreduce api.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181064#comment-13181064
]
Liyin Liang commented on MAPREDUCE-3619:
Before we make streaming work with both the old mapred API and the new
mapreduce API, should we fix the NPE first?
We can add null checking in PipeMapRed.MRErrorThread.incrCounter() and
setStatus() to avoid throwing NPE.
> Change streaming code to use new mapreduce api.
> ---
>
> Key: MAPREDUCE-3619
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3619
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/streaming, mrv2
>Affects Versions: 0.23.1
>Reporter: Liyin Liang
>
> If we run a streaming job with following python script as mapper or reducer,
> the job will throws NullPointerException.
> {code:}
> #!/usr/bin/python
> import sys,os
> class MyTask:
> def __init__(self, file=sys.stdin):
> self.file = file
> print >>sys.stderr, "reporter:counter:spam,disp_flag_record,0"
> print >>sys.stderr, "reporter:counter:spam,spam_record,0"
> def process(self):
> while True:
> line = self.file.readline()
> if not line:
> break;
> print line
> if __name__ == "__main__":
> task = MyTask()
> task.process()
> {code}
> Here is the NPE related log:
> 2011-12-22 14:14:06,310 WARN org.apache.hadoop.streaming.PipeMapRed:
> java.lang.NullPointerException
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.incrCounter(PipeMapRed.java:502)
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:444)
> This is because the above script's "print >>sys.stderr" will invoke
> reporter.incrCounter() during PipeMapper|PipeReducer.configure(). While we
> can not get reporter in configure() function.
> To fix this problem, we should change streaming code to use new-api. Then we
> can call context.getCounter() in Mapper|Reducer.setup() function.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3631) Corner case in headroom calculation
Corner case in headroom calculation --- Key: MAPREDUCE-3631 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3631 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Amol Kekre Fix For: 0.23.1 When there is a single queue and a large job fills up the whole cluster, a lost NM can lead to wrong headroom when all slots are taken up by reduces since at that point headroom isn't recomputed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3629) Remove sleep from MRAppMaster during app-finish.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amol Kekre updated MAPREDUCE-3629: -- Priority: Critical (was: Major) > Remove sleep from MRAppMaster during app-finish. > > > Key: MAPREDUCE-3629 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3629 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre >Priority: Critical > Fix For: 0.23.1 > > > MRAppMaster waits for 5 secs during app-finish, this was needed before we had > client-side redirection. This affects the app execution in that, AppMaster > will killed by the NM once NM gets > confirmation from RM. > AppMaster should go away immediately. Also, the done call to RM from AM > should be the last thing AM ever does. > Otherwise, today, JobHistory writing gets interrupted if AM gets killed by > the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3630) NullPointerException running teragen
[ https://issues.apache.org/jira/browse/MAPREDUCE-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amol Kekre updated MAPREDUCE-3630: -- Component/s: mrv2 Affects Version/s: 0.23.0 Fix Version/s: 0.23.1 > NullPointerException running teragen > > > Key: MAPREDUCE-3630 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3630 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre > Fix For: 0.23.1 > > > CMD = /grid/0/gs/gridre/yroot.omegab/share/hadoopcommon/bin/hadoop --config > /grid/0/gs/gridre/yroot.omegab/conf/hadoop/ > jar > /grid/0/gs/gridre/yroot.omegab/share/hadoopmapred/hadoop-mapreduce-examples-*.jar > teragen > -Dmapred.job.queue.name=audience -Dmapreduce.job.acl-view-job=* > -Dyarn.app.mapreduce.am.staging-dir=/user > -Dmapreduce.jobtracker.staging.root.dir=/user 1 teraInputDir > Error: > 11/09/21 21:59:59 INFO mapreduce.Job: map 50% reduce 0% > 11/09/21 22:00:00 INFO mapreduce.Job: Task Id : > attempt_1316132655177_0533_m_01_0, Status : FAILED > java.lang.NullPointerException > at > org.apache.hadoop.examples.terasort.TeraGen$SortGenMapper.cleanup(TeraGen.java:241) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:708) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:148) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:143) > 11/09/21 22:00:00 WARN mapreduce.Job: Error reading task output Server > returned HTTP response code: 400 for URL: > http://.../tasklog?plaintext=true&attemptid=attempt_1316132655177_0533_m_01_0&filter=stdout > 11/09/21 22:00:00 WARN mapreduce.Job: Error reading task output Server > returned HTTP response code: 400 for URL: > http://.../tasklog?plaintext=true&attemptid=attempt_1316132655177_0533_m_01_0&filter=stderr -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3630) NullPointerException running teragen
[ https://issues.apache.org/jira/browse/MAPREDUCE-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amol Kekre updated MAPREDUCE-3630: -- Priority: Critical (was: Major) > NullPointerException running teragen > > > Key: MAPREDUCE-3630 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3630 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre >Priority: Critical > Fix For: 0.23.1 > > > CMD = /grid/0/gs/gridre/yroot.omegab/share/hadoopcommon/bin/hadoop --config > /grid/0/gs/gridre/yroot.omegab/conf/hadoop/ > jar > /grid/0/gs/gridre/yroot.omegab/share/hadoopmapred/hadoop-mapreduce-examples-*.jar > teragen > -Dmapred.job.queue.name=audience -Dmapreduce.job.acl-view-job=* > -Dyarn.app.mapreduce.am.staging-dir=/user > -Dmapreduce.jobtracker.staging.root.dir=/user 1 teraInputDir > Error: > 11/09/21 21:59:59 INFO mapreduce.Job: map 50% reduce 0% > 11/09/21 22:00:00 INFO mapreduce.Job: Task Id : > attempt_1316132655177_0533_m_01_0, Status : FAILED > java.lang.NullPointerException > at > org.apache.hadoop.examples.terasort.TeraGen$SortGenMapper.cleanup(TeraGen.java:241) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:708) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:148) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:143) > 11/09/21 22:00:00 WARN mapreduce.Job: Error reading task output Server > returned HTTP response code: 400 for URL: > http://.../tasklog?plaintext=true&attemptid=attempt_1316132655177_0533_m_01_0&filter=stdout > 11/09/21 22:00:00 WARN mapreduce.Job: Error reading task output Server > returned HTTP response code: 400 for URL: > http://.../tasklog?plaintext=true&attemptid=attempt_1316132655177_0533_m_01_0&filter=stderr -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3630) NullPointerException running teragen
NullPointerException running teragen Key: MAPREDUCE-3630 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3630 Project: Hadoop Map/Reduce Issue Type: Task Reporter: Amol Kekre CMD = /grid/0/gs/gridre/yroot.omegab/share/hadoopcommon/bin/hadoop --config /grid/0/gs/gridre/yroot.omegab/conf/hadoop/ jar /grid/0/gs/gridre/yroot.omegab/share/hadoopmapred/hadoop-mapreduce-examples-*.jar teragen -Dmapred.job.queue.name=audience -Dmapreduce.job.acl-view-job=* -Dyarn.app.mapreduce.am.staging-dir=/user -Dmapreduce.jobtracker.staging.root.dir=/user 1 teraInputDir Error: 11/09/21 21:59:59 INFO mapreduce.Job: map 50% reduce 0% 11/09/21 22:00:00 INFO mapreduce.Job: Task Id : attempt_1316132655177_0533_m_01_0, Status : FAILED java.lang.NullPointerException at org.apache.hadoop.examples.terasort.TeraGen$SortGenMapper.cleanup(TeraGen.java:241) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:708) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:148) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:143) 11/09/21 22:00:00 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://.../tasklog?plaintext=true&attemptid=attempt_1316132655177_0533_m_01_0&filter=stdout 11/09/21 22:00:00 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://.../tasklog?plaintext=true&attemptid=attempt_1316132655177_0533_m_01_0&filter=stderr -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3629) Remove sleep from MRAppMaster during app-finish.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181047#comment-13181047 ] Amol Kekre commented on MAPREDUCE-3629: --- Vinod, Can you check this and comment (close if this is resolved) > Remove sleep from MRAppMaster during app-finish. > > > Key: MAPREDUCE-3629 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3629 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Amol Kekre > Fix For: 0.23.1 > > > MRAppMaster waits for 5 secs during app-finish, this was needed before we had > client-side redirection. This affects the app execution in that, AppMaster > will killed by the NM once NM gets > confirmation from RM. > AppMaster should go away immediately. Also, the done call to RM from AM > should be the last thing AM ever does. > Otherwise, today, JobHistory writing gets interrupted if AM gets killed by > the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3629) Remove sleep from MRAppMaster during app-finish.
Remove sleep from MRAppMaster during app-finish. Key: MAPREDUCE-3629 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3629 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Amol Kekre Fix For: 0.23.1 MRAppMaster waits for 5 secs during app-finish, this was needed before we had client-side redirection. This affects the app execution in that, AppMaster will killed by the NM once NM gets confirmation from RM. AppMaster should go away immediately. Also, the done call to RM from AM should be the last thing AM ever does. Otherwise, today, JobHistory writing gets interrupted if AM gets killed by the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3628) DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster. - Key: MAPREDUCE-3628 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3628 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Amol Kekre Priority: Critical Fix For: 0.23.1 DFSIO read throughput is decreased by 16% in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for running clients which are not kerberos authenticated
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181037#comment-13181037 ] Amol Kekre commented on MAPREDUCE-3380: --- any updates? > Token infrastructure for running clients which are not kerberos authenticated > - > > Key: MAPREDUCE-3380 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mr-am, mrv2 >Affects Versions: 0.23.0 >Reporter: Alejandro Abdelnur >Assignee: Mahadev konar >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3380.branch-0.23.patch, > MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, > MAPREDUCE-3380_v1.patch > > > The JobClient.getDelegationToken() method is returning NULL, this makes Oozie > fail when trying to get the delegation token to use it for starting a job. > What is seems to be happing is that Jobclient.getDelegationToken() calls > Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that > calls ResourceMgrDelegate.getDelegationToken(). And the last one is not > implemented. (Thanks Ahmed for tracing this in MR2 code) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3596) Sort benchmark got hang after completion of 99% map phase
[
https://issues.apache.org/jira/browse/MAPREDUCE-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated MAPREDUCE-3596:
--
Attachment: logs.tar.bz2
Should've seen the exception
anyway, sections of another run with the same exception. rm + am + nm which
caused the error.
> Sort benchmark got hang after completion of 99% map phase
> -
>
> Key: MAPREDUCE-3596
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3596
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mrv2
>Affects Versions: 0.23.0
>Reporter: Ravi Prakash
>Assignee: Vinod Kumar Vavilapalli
>Priority: Blocker
> Attachments: logs.tar.bz2, logs.tar.bz2
>
>
> Courtesy [~vinaythota]
> {quote}
> Ran sort benchmark couple of times and every time the job got hang after
> completion 99% map phase. There are some map tasks failed. Also it's not
> scheduled some of the pending map tasks.
> Cluster size is 350 nodes.
> Build Details:
> ==
> Compiled: Fri Dec 9 16:25:27 PST 2011 by someone from
> branches/branch-0.23/hadoop-common-project/hadoop-common
> ResourceManager version:revision 1212681 by someone source checksum
> on Fri Dec 9 16:52:07 PST 2011
> Hadoop version: revision 1212592 by someone Fri Dec 9 16:25:27 PST
> 2011
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3494) The RM should handle the graceful shutdown of the NM.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3494: - Component/s: mrv2 Please add 'mrv2' component for all relevant jiras, so I can track it as it becomes PA. Thanks. > The RM should handle the graceful shutdown of the NM. > - > > Key: MAPREDUCE-3494 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3494 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager, resourcemanager >Reporter: Ravi Teja Ch N V >Assignee: Devaraj K > Attachments: MAPREDUCE-3494.1.patch, MAPREDUCE-3494.2.patch, > MAPREDUCE-3494.patch > > > Instead of waiting for the NM expiry, RM should remove and handle the NM, > which is shutdown gracefully. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3626) Findbugs warning in ContainerRemoteLaunchEvent
[ https://issues.apache.org/jira/browse/MAPREDUCE-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3626: - Resolution: Duplicate Status: Resolved (was: Patch Available) Duplicate of MAPREDUCE-3622. > Findbugs warning in ContainerRemoteLaunchEvent > -- > > Key: MAPREDUCE-3626 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3626 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Arun C Murthy >Assignee: Arun C Murthy > Attachments: MAPREDUCE-3626.patch > > > Warning: > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent > defines equals but not hashCode -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3312) Make MR AM not send a stopContainer w/o corresponding start container
[ https://issues.apache.org/jira/browse/MAPREDUCE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180961#comment-13180961 ] Hadoop QA commented on MAPREDUCE-3312: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509624/MR-3312.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1552//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1552//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1552//console This message is automatically generated. > Make MR AM not send a stopContainer w/o corresponding start container > - > > Key: MAPREDUCE-3312 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3312 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Fix For: 0.23.1 > > Attachments: MR-3312.txt, MR-3312.txt > > > This is a follow on to MAPREDUCE-3274. It is possible, although rare, for > the MR AM to send a stop container before it sends a start container. This > needs to stop that from happening. If a stop is found first it should > prevent the start from being sent. It tries to do this, but only if the stop > is currently pending. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3543) Mavenize Gridmix.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Abdelnur updated MAPREDUCE-3543:
--
Attachment: MAPREDUCE-3543v1.patch
MAPREDUCE-3543v1.sh
*This is work in progress*
Script and patch to move Gridmix stuff to hadoop-tools.
Run first the script then apply the patch (works both in GIT and SVN).
The rumen stuff is moved into gridmix module.
There are dependencies in MR1 JobTracker and JobHistory and compilation fails
because of that:
{code}
[INFO] Compiling 139 source files to
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/target/classes
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/ClusterSummarizer.java:[26,31]
cannot find symbol
symbol : class JobTracker
location: package org.apache.hadoop.mapred
[ERROR]
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/tools/rumen/Pre21JobHistoryConstants.java:[23,45]
cannot find symbol
symbol : class JobHistory
location: package org.apache.hadoop.mapreduce.jobhistory
[ERROR]
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/tools/rumen/JobHistoryUtils.java:[30,45]
cannot find symbol
symbol : class JobHistory
location: package org.apache.hadoop.mapreduce.jobhistory
[ERROR]
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/tools/rumen/CurrentJHParser.java:[26,45]
cannot find symbol
symbol : class JobHistory
location: package org.apache.hadoop.mapreduce.jobhistory
[ERROR]
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/tools/rumen/TraceBuilder.java:[43,45]
cannot find symbol
symbol : class JobHistory
location: package org.apache.hadoop.mapreduce.jobhistory
[ERROR]
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/tools/rumen/Pre21JobHistoryConstants.java:[67,35]
cannot find symbol
symbol : variable JobHistory
location: class org.apache.hadoop.tools.rumen.Pre21JobHistoryConstants
[ERROR]
/Users/tucu/src/apache/hadoop/git/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/tools/rumen/JobHistoryUtils.java:[138,33]
cannot find symbol
symbol : variable JobHistory
location: class org.apache.hadoop.tools.rumen.JobHistoryUtils
[INFO] 7 errors
[INFO] -
[INFO]
[INFO] BUILD FAILURE
[INFO]
{code}
I assume the testcases will requires some minor tweaks like the streaming and
archives testcases. But cannot reach there as things don't compile.
I'm sure about the tests under contrib/gridmix/src/test/system, thus I didn't
move those yet.
> Mavenize Gridmix.
> -
>
> Key: MAPREDUCE-3543
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3543
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Reporter: Mahadev konar
>Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3543v1.patch, MAPREDUCE-3543v1.sh
>
>
> Gridmix codebase still resides in src/contrib and needs to be compiled via
> ant. We should move it to maven.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3299) Add AMInfo table to the AM job page
[ https://issues.apache.org/jira/browse/MAPREDUCE-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-3299: --- Attachment: MAPREDUCE-3299.patch A version with the web services and web services tests. The tests are failing at this point, but this can give everyone a jump start on looking at the code. > Add AMInfo table to the AM job page > --- > > Key: MAPREDUCE-3299 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3299 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Siddharth Seth >Assignee: Jonathan Eagles >Priority: Minor > Attachments: MAPREDUCE-3299.patch, MAPREDUCE-3299.patch, > MAPREDUCE-3299.patch, MAPREDUCE-3299.patch > > > JobHistory has a table to list all AMs. A similar table can be added to the > AM for info on past failed AMs and the current running one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3616) Thread pool for launching containers in MR AM not expanding as expected
[ https://issues.apache.org/jira/browse/MAPREDUCE-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180951#comment-13180951 ] Hadoop QA commented on MAPREDUCE-3616: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509621/MAPREDUCE-3616-20120105.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1551//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1551//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1551//console This message is automatically generated. > Thread pool for launching containers in MR AM not expanding as expected > --- > > Key: MAPREDUCE-3616 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3616 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mr-am, performance >Affects Versions: 0.23.1 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3616-20120105.txt > > > Found this while running some benchmarks on 350 nodes. The thread pool stays > at 60 for a long time and only expands to 350 towards the fag end of the job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3312) Make MR AM not send a stopContainer w/o corresponding start container
[ https://issues.apache.org/jira/browse/MAPREDUCE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-3312: --- Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) > Make MR AM not send a stopContainer w/o corresponding start container > - > > Key: MAPREDUCE-3312 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3312 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Fix For: 0.23.1 > > Attachments: MR-3312.txt, MR-3312.txt > > > This is a follow on to MAPREDUCE-3274. It is possible, although rare, for > the MR AM to send a stop container before it sends a start container. This > needs to stop that from happening. If a stop is found first it should > prevent the start from being sent. It tries to do this, but only if the stop > is currently pending. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3312) Make MR AM not send a stopContainer w/o corresponding start container
[ https://issues.apache.org/jira/browse/MAPREDUCE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-3312: --- Attachment: MR-3312.txt This version of the patch will not change the state machine, instead it updates the ContaienrLauncherImpl to keep track of state for each container and do the right thing. This removes the need for ContainerRemoteLaunchEvent.equals which appears to have already been changed from its original implementation. Originally it was used to remove all related events from the queue. > Make MR AM not send a stopContainer w/o corresponding start container > - > > Key: MAPREDUCE-3312 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3312 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Fix For: 0.23.1 > > Attachments: MR-3312.txt, MR-3312.txt > > > This is a follow on to MAPREDUCE-3274. It is possible, although rare, for > the MR AM to send a stop container before it sends a start container. This > needs to stop that from happening. If a stop is found first it should > prevent the start from being sent. It tries to do this, but only if the stop > is currently pending. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3616) Thread pool for launching containers in MR AM not expanding as expected
[ https://issues.apache.org/jira/browse/MAPREDUCE-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3616: --- Status: Patch Available (was: Open) > Thread pool for launching containers in MR AM not expanding as expected > --- > > Key: MAPREDUCE-3616 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3616 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mr-am, performance >Affects Versions: 0.23.1 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3616-20120105.txt > > > Found this while running some benchmarks on 350 nodes. The thread pool stays > at 60 for a long time and only expands to 350 towards the fag end of the job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3616) Thread pool for launching containers in MR AM not expanding as expected
[ https://issues.apache.org/jira/browse/MAPREDUCE-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3616: --- Attachment: MAPREDUCE-3616-20120105.txt This should expand the thread pool correctly. Also added a couple of tests for the way the pool is expanded and how limits on the pool-size are handled. > Thread pool for launching containers in MR AM not expanding as expected > --- > > Key: MAPREDUCE-3616 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3616 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mr-am, performance >Affects Versions: 0.23.1 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3616-20120105.txt > > > Found this while running some benchmarks on 350 nodes. The thread pool stays > at 60 for a long time and only expands to 350 towards the fag end of the job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3627) ClusterSetup docs for permissions on mapreduce.jobhistory.done-dir don't match jobhistory server startup
ClusterSetup docs for permissions on mapreduce.jobhistory.done-dir don't match jobhistory server startup Key: MAPREDUCE-3627 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3627 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Priority: Critical The ClusterSetup docs for 0.23 list the permissions on mapreduce.jobhistory.done-dir should be drwxr-x---, but the job history server which will create that directory for you if its not there sets them to be 0770. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3389) MRApps loads the 'mrapp-generated-classpath' file with classpath from the build machine
[ https://issues.apache.org/jira/browse/MAPREDUCE-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-3389: -- Fix Version/s: (was: 0.24.0) > MRApps loads the 'mrapp-generated-classpath' file with classpath from the > build machine > --- > > Key: MAPREDUCE-3389 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3389 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.23.1, 0.24.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur >Priority: Critical > Labels: bigtop > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3389v1.patch, MAPREDUCE-3389v2.patch > > > The 'mrapp-generated-classpath' file contains the classpath from where Hadoop > was build. This classpath is not useful under any circumstances. > For example the content of the 'mrapp-generated-classpath' in my dev > environment is: > /Users/tucu/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/tucu/.m2/repository/asm/asm/3.2/asm-3.2.jar:/Users/tucu/.m2/repository/com/cenqua/clover/clover/3.0.2/clover-3.0.2.jar:/Users/tucu/.m2/repository/com/google/guava/guava/r09/guava-r09.jar:/Users/tucu/.m2/repository/com/google/inject/guice/2.0/guice-2.0.jar:/Users/tucu/.m2/repository/com/google/inject/extensions/guice-servlet/2.0/guice-servlet-2.0.jar:/Users/tucu/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-json/1.8/jersey-json-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/Users/tucu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/tucu/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/Users/tucu/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/Users/tucu/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/Users/tucu/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/tucu/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/tucu/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/Users/tucu/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/tucu/.m2/repository/commons-daemon/commons-daemon/1.0.3/commons-daemon-1.0.3.jar:/Users/tucu/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/tucu/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/tucu/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/tucu/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/Users/tucu/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar:/Users/tucu/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/Users/tucu/.m2/repository/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar:/Users/tucu/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar:/Users/tucu/.m2/repository/hsqldb/hsqldb/1.8.0.7/hsqldb-1.8.0.7.jar:/Users/tucu/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/tucu/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:/Users/tucu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/Users/tucu/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/Users/tucu/.m2/repository/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar:/Users/tucu/.m2/repository/jdiff/jdiff/1.0.9/jdiff-1.0.9.jar:/Users/tucu/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/Users/tucu/.m2/repository/junit/junit/4.8.2/junit-4.8.2.jar:/Users/tucu/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar:/Users/tucu/.m2/repository/net/java/dev/jets3t/jets3t/0.6.1/jets3t-0.6.1.jar:/Users/tucu/.m2/repository/net/sf/kosmosfs/kfs/0.3/kfs-0.3.jar:/Users/tucu/.m2/repository/org/apache/avro/avro/1.5.3/avro-1.5.3.jar:/Users/tucu/.m2/repository/org/apache/avro/avro-ipc/1.5.3/avro-ipc-1.5.3.jar:/Users/tucu/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/tucu/src/apache/hadoop/git/hadoop-common-project/hadoop-annotations/target/hadoop-annotations-0.24.0-SNAPSHOT.jar:/Users/tucu/src/apache/hadoop/git/hadoop-common-project/hadoop-auth/target/hadoop-auth-0.24.0-SNAPSHOT.jar:/Users/tucu/src/apache/hadoop/git/hadoop-common-project/hadoop-common/target/hadoop-common-0.24.0-SNAPSHOT.jar:/Users/tucu/src/apache/hadoop/git/hadoop-common-project/hadoop-common/target/hadoop-common-0.24.0-SNAPSHOT-tests.jar:/Users/tucu/src/a
[jira] [Updated] (MAPREDUCE-3544) gridmix build is broken, requires hadoop-archives to be added as ivy dependency
[ https://issues.apache.org/jira/browse/MAPREDUCE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-3544: -- Fix Version/s: (was: 0.24.0) > gridmix build is broken, requires hadoop-archives to be added as ivy > dependency > --- > > Key: MAPREDUCE-3544 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3544 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, tools/rumen >Affects Versions: 0.23.1, 0.24.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3544.patch > > > Having moved HAR/HadoopArchives to common/tools makes gridmix to fail as > HadoopArchives is not in the mr1 classpath anymore. > hadoop-archives artifact should be added to gridmix dependencies -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3557) MR1 test fail to compile because of missing hadoop-archives dependency
[ https://issues.apache.org/jira/browse/MAPREDUCE-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-3557: -- Fix Version/s: (was: 0.24.0) > MR1 test fail to compile because of missing hadoop-archives dependency > -- > > Key: MAPREDUCE-3557 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3557 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.23.1, 0.24.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3557.patch > > > MAPREDUCE-3544 added hadoop-archives as dependency to gridmix and raid, but > missed to add it to the main ivy.xml for the MR1 testcases thus the ant > target 'compile-mapred-test' fails. > I was under the impression that this stuff was not used anymore but trunk is > failing on that target. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3415) improve MiniMRYarnCluster & DistributedShell JAR resolution
[ https://issues.apache.org/jira/browse/MAPREDUCE-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-3415: -- Fix Version/s: (was: 0.24.0) > improve MiniMRYarnCluster & DistributedShell JAR resolution > > > Key: MAPREDUCE-3415 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3415 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3415.patch, MAPREDUCE-3415.patch, > MAPREDUCE-3415.patch, MAPREDUCE-3415.patch > > > Current JAR resolution assumes the following: > # The class used for JAR lookup is effectively in a JAR > # A System property is set for testing with the location of the JAR > The problem with #1 is that in some cases (when using the class in the same > Maven module where the class is, the class is not in a JAR but in a directory > 'target/test-classes'). > The problem with #2 is the JAR does not exists at the time of running the > test (packaging comes after test and we are not doing integration testing yet > thus won't work) > In addition, this is required for streaming testcases, to have the JAR with > streaming classes for testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3500) MRJobConfig creates an LD_LIBRARY_PATH using the platform ARCH
[
https://issues.apache.org/jira/browse/MAPREDUCE-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Abdelnur updated MAPREDUCE-3500:
--
Fix Version/s: (was: 0.24.0)
> MRJobConfig creates an LD_LIBRARY_PATH using the platform ARCH
> --
>
> Key: MAPREDUCE-3500
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3500
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3500.patch
>
>
> With HADOOP-7874 we are removing the arch from the java.library.path.
> The LD_LIBRARY_PATH being set should not include the ARCH.
> {code}
> public static final String DEFAULT_MAPRED_ADMIN_USER_ENV =
> "LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native/" +
> PlatformName.getPlatformName();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3626) Findbugs warning in ContainerRemoteLaunchEvent
[ https://issues.apache.org/jira/browse/MAPREDUCE-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3626: - Attachment: MAPREDUCE-3626.patch Straight-fwd fix. > Findbugs warning in ContainerRemoteLaunchEvent > -- > > Key: MAPREDUCE-3626 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3626 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Arun C Murthy >Assignee: Arun C Murthy > Attachments: MAPREDUCE-3626.patch > > > Warning: > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent > defines equals but not hashCode -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3626) Findbugs warning in ContainerRemoteLaunchEvent
[ https://issues.apache.org/jira/browse/MAPREDUCE-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3626: - Status: Patch Available (was: Open) > Findbugs warning in ContainerRemoteLaunchEvent > -- > > Key: MAPREDUCE-3626 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3626 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Arun C Murthy >Assignee: Arun C Murthy > Attachments: MAPREDUCE-3626.patch > > > Warning: > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent > defines equals but not hashCode -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3596) Sort benchmark got hang after completion of 99% map phase
[
https://issues.apache.org/jira/browse/MAPREDUCE-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180877#comment-13180877
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3596:
I see an exception in RM from the attached logs (rm1) which points to the fixed
issue MAPREDUCE-3530. Will need newer set of logs if this is still an issue.
> Sort benchmark got hang after completion of 99% map phase
> -
>
> Key: MAPREDUCE-3596
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3596
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mrv2
>Affects Versions: 0.23.0
>Reporter: Ravi Prakash
>Assignee: Vinod Kumar Vavilapalli
>Priority: Blocker
> Attachments: logs.tar.bz2
>
>
> Courtesy [~vinaythota]
> {quote}
> Ran sort benchmark couple of times and every time the job got hang after
> completion 99% map phase. There are some map tasks failed. Also it's not
> scheduled some of the pending map tasks.
> Cluster size is 350 nodes.
> Build Details:
> ==
> Compiled: Fri Dec 9 16:25:27 PST 2011 by someone from
> branches/branch-0.23/hadoop-common-project/hadoop-common
> ResourceManager version:revision 1212681 by someone source checksum
> on Fri Dec 9 16:52:07 PST 2011
> Hadoop version: revision 1212592 by someone Fri Dec 9 16:25:27 PST
> 2011
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3603) Add Web UI to MR2 Fair Scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180847#comment-13180847 ] Patrick Wendell commented on MAPREDUCE-3603: This will probably be a good amount of work, so I'm making a standalone ticket (was under MAPREDUCE-3600). It will require writing Hamlet code to plugin to the existing Scheduler view of the RM GUI. > Add Web UI to MR2 Fair Scheduler > > > Key: MAPREDUCE-3603 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3603 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: scheduler >Reporter: Patrick Wendell >Assignee: Patrick Wendell > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180843#comment-13180843 ] Hudson commented on MAPREDUCE-3617: --- Integrated in Hadoop-Mapreduce-0.23-Commit #358 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/358/]) Merge -c 1227808 from trunk to branch-0.23 to fix MAPREDUCE-3617.emoved wrong default value for yarn.resourcemanager.principal and yarn.nodemanager.principal. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227810 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3603) Add Web UI to MR2 Fair Scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated MAPREDUCE-3603: --- Issue Type: New Feature (was: Sub-task) Parent: (was: MAPREDUCE-3451) > Add Web UI to MR2 Fair Scheduler > > > Key: MAPREDUCE-3603 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3603 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: scheduler >Reporter: Patrick Wendell >Assignee: Patrick Wendell > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3548) write unit tests for web services for mapreduce app master and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180834#comment-13180834 ] Hudson commented on MAPREDUCE-3548: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1523 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1523/]) MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227801 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/ConfInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/CounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskAttemptCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/JAXBContextResolver.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesAttempts.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobConf.java * /hadoop/common/trunk/hadoop-mapreduce-project/
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180835#comment-13180835 ] Hudson commented on MAPREDUCE-3617: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1523 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1523/]) MAPREDUCE-3617. Removed wrong default value for yarn.resourcemanager.principal and yarn.nodemanager.principal. Contributed by Jonathan Eagles. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227808 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180830#comment-13180830 ] Hudson commented on MAPREDUCE-3326: --- Integrated in Hadoop-Mapreduce-0.23-Commit #357 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/357/]) Merge -c 1227788 from trunk to branch-0.23 to fix MAPREDUCE-3326. Added detailed information about queue's to the CapacityScheduler web-ui. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227794 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ResponseInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySched.java > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3548) write unit tests for web services for mapreduce app master and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180832#comment-13180832 ] Hudson commented on MAPREDUCE-3548: --- Integrated in Hadoop-Mapreduce-0.23-Commit #357 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/357/]) Merge -c 1227801 from trunk to branch-0.23 to fix MAPREDUCE-3548. MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227802 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/AppInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/ConfInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/CounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskAttemptCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/JAXBContextResolver.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java * /hadoop/common/branches/branch-0.23/hado
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180820#comment-13180820 ] Hudson commented on MAPREDUCE-3326: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1522 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1522/]) MAPREDUCE-3326. Added detailed information about queue's to the CapacityScheduler web-ui. Contributed by Jason Lowe. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227788 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ResponseInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySched.java > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180813#comment-13180813 ] Hudson commented on MAPREDUCE-3617: --- Integrated in Hadoop-Hdfs-trunk-Commit #1576 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1576/]) MAPREDUCE-3617. Removed wrong default value for yarn.resourcemanager.principal and yarn.nodemanager.principal. Contributed by Jonathan Eagles. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227808 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180814#comment-13180814 ] Hudson commented on MAPREDUCE-3617: --- Integrated in Hadoop-Common-0.23-Commit #347 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/347/]) Merge -c 1227808 from trunk to branch-0.23 to fix MAPREDUCE-3617.emoved wrong default value for yarn.resourcemanager.principal and yarn.nodemanager.principal. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227810 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180811#comment-13180811 ] Hudson commented on MAPREDUCE-3617: --- Integrated in Hadoop-Hdfs-0.23-Commit #337 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/337/]) Merge -c 1227808 from trunk to branch-0.23 to fix MAPREDUCE-3617.emoved wrong default value for yarn.resourcemanager.principal and yarn.nodemanager.principal. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227810 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3548) write unit tests for web services for mapreduce app master and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180805#comment-13180805 ] Hudson commented on MAPREDUCE-3548: --- Integrated in Hadoop-Common-trunk-Commit #1503 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1503/]) MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227801 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/ConfInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/CounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskAttemptCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/JAXBContextResolver.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesAttempts.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobConf.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180806#comment-13180806 ] Hudson commented on MAPREDUCE-3617: --- Integrated in Hadoop-Common-trunk-Commit #1503 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1503/]) MAPREDUCE-3617. Removed wrong default value for yarn.resourcemanager.principal and yarn.nodemanager.principal. Contributed by Jonathan Eagles. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227808 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3354) JobHistoryServer should be started by bin/mapred and not by bin/yarn
[ https://issues.apache.org/jira/browse/MAPREDUCE-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180807#comment-13180807 ] Jonathan Eagles commented on MAPREDUCE-3354: Thanks, Arun. Will take care of this. > JobHistoryServer should be started by bin/mapred and not by bin/yarn > > > Key: MAPREDUCE-3354 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3354 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3354.patch, MAPREDUCE-3354.patch, > MAPREDUCE-3354.patch, MAPREDUCE-3354.patch, MAPREDUCE-3354.patch > > > JobHistoryServer belongs to mapreduce land. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3548) write unit tests for web services for mapreduce app master and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180804#comment-13180804 ] Hudson commented on MAPREDUCE-3548: --- Integrated in Hadoop-Common-0.23-Commit #346 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/346/]) Merge -c 1227801 from trunk to branch-0.23 to fix MAPREDUCE-3548. MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227802 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/AppInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/ConfInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/CounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskAttemptCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/JAXBContextResolver.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-map
[jira] [Commented] (MAPREDUCE-3548) write unit tests for web services for mapreduce app master and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180800#comment-13180800 ] Hudson commented on MAPREDUCE-3548: --- Integrated in Hadoop-Hdfs-trunk-Commit #1575 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1575/]) MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227801 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/ConfInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/CounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskAttemptCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskCounterInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/JAXBContextResolver.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesAttempts.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobConf.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-map
[jira] [Commented] (MAPREDUCE-3548) write unit tests for web services for mapreduce app master and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180798#comment-13180798 ] Hudson commented on MAPREDUCE-3548: --- Integrated in Hadoop-Hdfs-0.23-Commit #336 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/336/]) Merge -c 1227801 from trunk to branch-0.23 to fix MAPREDUCE-3548. MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227802 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/AppInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/ConfInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/CounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskAttemptCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobTaskCounterInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskAttemptsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/TaskInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/JAXBContextResolver.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapredu
[jira] [Updated] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3617: - Resolution: Fixed Fix Version/s: 0.23.0 Target Version/s: 0.23.1 (was: 0.24.0, 0.23.1) Status: Resolved (was: Patch Available) I just committed this. Thanks Jonathan! I've opened MAPREDUCE-3626 to track the unrelated findbugs' warning. > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[
https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180793#comment-13180793
]
Hudson commented on MAPREDUCE-1744:
---
Integrated in Hadoop-Mapreduce-0.23-Commit #356 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/356/])
Merge -r 1227237:1227238 from trunk to branch. FIXES: MAPREDUCE-1744
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227789
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/filecache/DistributedCache.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMROldApiJobs.java
> DistributedCache creates its own FileSytem instance when adding a
> file/archive to the path
> --
>
> Key: MAPREDUCE-1744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Dick King
>Assignee: Dick King
> Fix For: 0.23.1, 1.0.0
>
> Attachments: BZ-3503564--2010-05-06.patch,
> MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744-0.23-trunk.patch,
> MAPREDUCE-1744.patch, h1744.patch, mapred-1744-1.patch, mapred-1744-2.patch,
> mapred-1744-3.patch, mapred-1744.patch
>
>
> According to the contract of {{UserGroupInformation.doAs()}} the only
> required operations within the {{doAs()}} block are the
> creation of a {{JobClient}} or getting a {{FileSystem}} .
> The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a
> {{FileSystem}} instance outside of the {{doAs()}} block,
> this {{FileSystem}} instance is not in the scope of the proxy user but of the
> superuser and permissions may make the method
> fail.
> One option is to overload the methods above to receive a filesystem.
> Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block,
> for this it would be required to have the proxy
> user set in the passed configuration.
> The second option seems nicer, but I don't know if the proxy user is as a
> property in the jobconf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3626) Findbugs warning in ContainerRemoteLaunchEvent
Findbugs warning in ContainerRemoteLaunchEvent -- Key: MAPREDUCE-3626 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3626 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Arun C Murthy Assignee: Arun C Murthy Warning: org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent defines equals but not hashCode -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost nodes in the UI.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3360: - Status: Open (was: Patch Available) Can you please look at the findbugs' warning? Also, can we ensure this information is available via the web-services api as well? Thanks! > Provide information about lost nodes in the UI. > --- > > Key: MAPREDUCE-3360 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 > Environment: NA >Reporter: Bhallamudi Venkata Siva Kamesh >Assignee: Bhallamudi Venkata Siva Kamesh >Priority: Critical > Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, > MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, > lostNodes.png > > > Currently there is no information provided about *lost nodes*. Provide > information in the UI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3354) JobHistoryServer should be started by bin/mapred and not by bin/yarn
[ https://issues.apache.org/jira/browse/MAPREDUCE-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3354: - Target Version/s: 0.23.1 (was: 0.24.0, 0.23.1) Affects Version/s: (was: 0.23.1) (was: 0.24.0) 0.23.0 Status: Open (was: Patch Available) Jon, if you don't mind, can you please update the docs as well? Thanks. > JobHistoryServer should be started by bin/mapred and not by bin/yarn > > > Key: MAPREDUCE-3354 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3354 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3354.patch, MAPREDUCE-3354.patch, > MAPREDUCE-3354.patch, MAPREDUCE-3354.patch, MAPREDUCE-3354.patch > > > JobHistoryServer belongs to mapreduce land. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180789#comment-13180789 ] Hudson commented on MAPREDUCE-3326: --- Integrated in Hadoop-Hdfs-0.23-Commit #335 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/335/]) Merge -c 1227788 from trunk to branch-0.23 to fix MAPREDUCE-3326. Added detailed information about queue's to the CapacityScheduler web-ui. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227794 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ResponseInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySched.java > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3548) write unit tests for web services for mapreduce app master and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3548: - Resolution: Fixed Fix Version/s: 0.23.1 Target Version/s: 0.23.1 (was: 0.23.0) Status: Resolved (was: Patch Available) I just committed this. Thanks Thomas! > write unit tests for web services for mapreduce app master and job history > server > - > > Key: MAPREDUCE-3548 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3548 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3548.patch, MAPREDUCE-3548.patch > > > write more unit tests for mapreduce application master and job history server > web services added in MAPREDUCE-2863 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[
https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180785#comment-13180785
]
Hudson commented on MAPREDUCE-1744:
---
Integrated in Hadoop-Common-0.23-Commit #345 (See
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/345/])
Merge -r 1227237:1227238 from trunk to branch. FIXES: MAPREDUCE-1744
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227789
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/filecache/DistributedCache.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMROldApiJobs.java
> DistributedCache creates its own FileSytem instance when adding a
> file/archive to the path
> --
>
> Key: MAPREDUCE-1744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Dick King
>Assignee: Dick King
> Fix For: 0.23.1, 1.0.0
>
> Attachments: BZ-3503564--2010-05-06.patch,
> MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744-0.23-trunk.patch,
> MAPREDUCE-1744.patch, h1744.patch, mapred-1744-1.patch, mapred-1744-2.patch,
> mapred-1744-3.patch, mapred-1744.patch
>
>
> According to the contract of {{UserGroupInformation.doAs()}} the only
> required operations within the {{doAs()}} block are the
> creation of a {{JobClient}} or getting a {{FileSystem}} .
> The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a
> {{FileSystem}} instance outside of the {{doAs()}} block,
> this {{FileSystem}} instance is not in the scope of the proxy user but of the
> superuser and permissions may make the method
> fail.
> One option is to overload the methods above to receive a filesystem.
> Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block,
> for this it would be required to have the proxy
> user set in the passed configuration.
> The second option seems nicer, but I don't know if the proxy user is as a
> property in the jobconf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180784#comment-13180784 ] Hudson commented on MAPREDUCE-3326: --- Integrated in Hadoop-Common-0.23-Commit #345 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/345/]) Merge -c 1227788 from trunk to branch-0.23 to fix MAPREDUCE-3326. Added detailed information about queue's to the CapacityScheduler web-ui. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227794 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ResponseInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySched.java > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180781#comment-13180781 ] Hudson commented on MAPREDUCE-3326: --- Integrated in Hadoop-Common-trunk-Commit #1502 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1502/]) MAPREDUCE-3326. Added detailed information about queue's to the CapacityScheduler web-ui. Contributed by Jason Lowe. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227788 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ResponseInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySched.java > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180779#comment-13180779 ] Jonathan Eagles commented on MAPREDUCE-3617: Findbugs is unrelated to this patch. No tests added for this change in the default yarn resource values > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3625) CapacityScheduler web-ui display of queue's used capacity is broken
[ https://issues.apache.org/jira/browse/MAPREDUCE-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3625: - Description: The display of the queue's used capacity at runtime is broken because it display's 'used' relative to the queue's capacity and not the parent's capacity as shown in the above attachment. The display should be relative to parent's capacity and not leaf queues as everything else in the display is relative to parent's capacity. was: The resource manager web ui page for scheduler doesn't have all the information about the configuration like the jobtracker page used to have. The things it seems to show you are the current queues - each queues used, set, and max percent and then what apps are running in that queue. It doesn't list any of yarn.scheduler.capacity.maximum-applications, yarn.scheduler.capacity.maximum-am-resource-percent, yarn.scheduler.capacity..user-limit-factor, yarn.scheduler.capacity..minimum-user-limit-percent, queue state, active users and percent used by user Affects Version/s: (was: 0.24.0) Reporter: Arun C Murthy (was: Thomas Graves) Fix Version/s: (was: 0.23.1) > CapacityScheduler web-ui display of queue's used capacity is broken > --- > > Key: MAPREDUCE-3625 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3625 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Jason Lowe >Priority: Critical > > The display of the queue's used capacity at runtime is broken because it > display's 'used' relative to the queue's capacity and not the parent's > capacity as shown in the above attachment. > The display should be relative to parent's capacity and not leaf queues as > everything else in the display is relative to parent's capacity. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180777#comment-13180777 ] Hudson commented on MAPREDUCE-3326: --- Integrated in Hadoop-Hdfs-trunk-Commit #1574 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1574/]) MAPREDUCE-3326. Added detailed information about queue's to the CapacityScheduler web-ui. Contributed by Jason Lowe. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227788 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ResponseInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySched.java > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[
https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180776#comment-13180776
]
Hudson commented on MAPREDUCE-1744:
---
Integrated in Hadoop-Hdfs-0.23-Commit #334 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/334/])
Merge -r 1227237:1227238 from trunk to branch. FIXES: MAPREDUCE-1744
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227789
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/filecache/DistributedCache.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMROldApiJobs.java
> DistributedCache creates its own FileSytem instance when adding a
> file/archive to the path
> --
>
> Key: MAPREDUCE-1744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Dick King
>Assignee: Dick King
> Fix For: 0.23.1, 1.0.0
>
> Attachments: BZ-3503564--2010-05-06.patch,
> MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744-0.23-trunk.patch,
> MAPREDUCE-1744.patch, h1744.patch, mapred-1744-1.patch, mapred-1744-2.patch,
> mapred-1744-3.patch, mapred-1744.patch
>
>
> According to the contract of {{UserGroupInformation.doAs()}} the only
> required operations within the {{doAs()}} block are the
> creation of a {{JobClient}} or getting a {{FileSystem}} .
> The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a
> {{FileSystem}} instance outside of the {{doAs()}} block,
> this {{FileSystem}} instance is not in the scope of the proxy user but of the
> superuser and permissions may make the method
> fail.
> One option is to overload the methods above to receive a filesystem.
> Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block,
> for this it would be required to have the proxy
> user set in the passed configuration.
> The second option seems nicer, but I don't know if the proxy user is as a
> property in the jobconf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180775#comment-13180775 ] Hadoop QA commented on MAPREDUCE-3617: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509583/MAPREDUCE-3617.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1548//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1548//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1548//console This message is automatically generated. > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3326: - Resolution: Fixed Fix Version/s: 0.23.1 Target Version/s: 0.23.1 (was: 0.24.0, 0.23.1) Status: Resolved (was: Patch Available) I just committed this. Thanks Jason (and Tom for reviewing)! > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3625) CapacityScheduler web-ui display of queue's used capacity is broken
CapacityScheduler web-ui display of queue's used capacity is broken --- Key: MAPREDUCE-3625 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3625 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Thomas Graves Assignee: Jason Lowe Priority: Critical Fix For: 0.23.1 The resource manager web ui page for scheduler doesn't have all the information about the configuration like the jobtracker page used to have. The things it seems to show you are the current queues - each queues used, set, and max percent and then what apps are running in that queue. It doesn't list any of yarn.scheduler.capacity.maximum-applications, yarn.scheduler.capacity.maximum-am-resource-percent, yarn.scheduler.capacity..user-limit-factor, yarn.scheduler.capacity..minimum-user-limit-percent, queue state, active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[
https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180769#comment-13180769
]
Alejandro Abdelnur commented on MAPREDUCE-1744:
---
Just committed to branch-0.23 as well.
> DistributedCache creates its own FileSytem instance when adding a
> file/archive to the path
> --
>
> Key: MAPREDUCE-1744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Dick King
>Assignee: Dick King
> Fix For: 0.23.1, 1.0.0
>
> Attachments: BZ-3503564--2010-05-06.patch,
> MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744-0.23-trunk.patch,
> MAPREDUCE-1744.patch, h1744.patch, mapred-1744-1.patch, mapred-1744-2.patch,
> mapred-1744-3.patch, mapred-1744.patch
>
>
> According to the contract of {{UserGroupInformation.doAs()}} the only
> required operations within the {{doAs()}} block are the
> creation of a {{JobClient}} or getting a {{FileSystem}} .
> The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a
> {{FileSystem}} instance outside of the {{doAs()}} block,
> this {{FileSystem}} instance is not in the scope of the proxy user but of the
> superuser and permissions may make the method
> fail.
> One option is to overload the methods above to receive a filesystem.
> Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block,
> for this it would be required to have the proxy
> user set in the passed configuration.
> The second option seems nicer, but I don't know if the proxy user is as a
> property in the jobconf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180764#comment-13180764 ] Arun C Murthy commented on MAPREDUCE-3326: -- On second thoughts, let's get this one in and open a new jira for the CS bug. > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost nodes in the UI.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180760#comment-13180760 ] Jason Lowe commented on MAPREDUCE-3360: --- +1, looks good to me. > Provide information about lost nodes in the UI. > --- > > Key: MAPREDUCE-3360 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 > Environment: NA >Reporter: Bhallamudi Venkata Siva Kamesh >Assignee: Bhallamudi Venkata Siva Kamesh >Priority: Critical > Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, > MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, > lostNodes.png > > > Currently there is no information provided about *lost nodes*. Provide > information in the UI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180756#comment-13180756 ] Hadoop QA commented on MAPREDUCE-3326: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509595/Screen+shot+2012-01-05+at+11.23.26+AM.png against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1549//console This message is automatically generated. > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3615) mapred ant test failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180752#comment-13180752 ] Hudson commented on MAPREDUCE-3615: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1520 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1520/]) MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java > mapred ant test failures > > > Key: MAPREDUCE-3615 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3615 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3615.patch > > > The following mapred ant tests are failing. This started on December 22nd. > [junit] Running org.apache.hadoop.mapred.TestTrackerBlacklistAcrossJobs > [junit] Running org.apache.hadoop.mapred.TestMiniMRDFSSort > [junit] Running org.apache.hadoop.mapred.TestBadRecords > [junit] Running org.apache.hadoop.mapred.TestClusterMRNotification > [junit] Running org.apache.hadoop.mapred.TestDebugScript > [junit] Running org.apache.hadoop.mapred.TestJobCleanup > [junit] Running org.apache.hadoop.mapred.TestJobClient > [junit] Running org.apache.hadoop.mapred.TestJobHistory > [junit] Running org.apache.hadoop.mapred.TestJobInProgressListener > [junit] Running org.apache.hadoop.mapred.TestJobKillAndFail > [junit] Running org.apache.hadoop.mapred.TestJvmReuse > [junit] Running org.apache.hadoop.mapred.TestKillSubProcesses > [junit] Running org.apache.hadoop.mapred.TestNodeRefresh > [junit] Running org.apache.hadoop.mapred.TestSetupAndCleanupFailure > [junit] Running org.apache.hadoop.mapred.TestTaskFail > [junit] Running org.apache.hadoop.mapred.TestTaskOutputSize > [junit] Running org.apache.hadoop.mapred.TestTaskTrackerSlotManagement > [junit] Running org.apache.hadoop.mapreduce.TestMRJobClient > [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob > [junit] Running > org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180748#comment-13180748 ] Thomas Graves commented on MAPREDUCE-3326: -- docs will be handled under MAPREDUCE-3549 - I added a comment. > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3549) write api documentation for web service apis for RM, NM, mapreduce app master, and job history server
[ https://issues.apache.org/jira/browse/MAPREDUCE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180746#comment-13180746 ] Thomas Graves commented on MAPREDUCE-3549: -- include capacity scheduler changes made under MAPREDUCE-3326. > write api documentation for web service apis for RM, NM, mapreduce app > master, and job history server > - > > Key: MAPREDUCE-3549 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3549 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > > write api documentation for web service apis for RM, NM, mapreduce app > master, and job history server. web services were added in MAPREDUCE-2863 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3326: - Attachment: Screen shot 2012-01-05 at 11.23.26 AM.png One nit - an existing bug is that the display of the queue's used capacity at runtime is broken because it display's 'used' relative to the queue's capacity and not the parent's capacity as shown in the above attachment. I think the display should be relative to parent's capacity and not leaf queues as everything else in the display is relative to parent's capacity. Jason, if you don't mind can we fix that alongwith this patch? Thanks! > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-3326.patch, Screen shot 2012-01-05 at 11.23.26 > AM.png > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180739#comment-13180739 ] Jason Lowe commented on MAPREDUCE-3326: --- Sorry I forgot to mention that the patch is implemented in terms of the web dao objects, so the web services have access to the same information as the web page. The web services tests were updated as part of the patch to verify this. I was going to update the web services documentation, but Tom informed me that part hasn't been written yet. Tom said he'd update the docs to describe the new fields in the web dao objects before the first draft is submitted. If those docs go in before this patch then I can update the patch, or we can file a JIRA to track updating the docs. > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-3326.patch > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3624) bin/yarn script adds jdk tools.jar to the classpath.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180738#comment-13180738
]
Hadoop QA commented on MAPREDUCE-3624:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509578/MAPREDUCE-3624.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1546//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1546//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1546//console
This message is automatically generated.
> bin/yarn script adds jdk tools.jar to the classpath.
>
>
> Key: MAPREDUCE-3624
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3624
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Mahadev konar
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3624.patch
>
>
> Thanks to Roman for pointing it out. Looks like we have the following lines
> in bin/yarn:
> {code}
> CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar
> {code}
> We dont really have a dependency on the tools jar. We should remove this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3299) Add AMInfo table to the AM job page
[ https://issues.apache.org/jira/browse/MAPREDUCE-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180735#comment-13180735 ] Jonathan Eagles commented on MAPREDUCE-3299: Thanks for taking a looks guys. Will add this as part of this JIRA so they don't get out of sync. > Add AMInfo table to the AM job page > --- > > Key: MAPREDUCE-3299 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3299 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Siddharth Seth >Assignee: Jonathan Eagles >Priority: Minor > Attachments: MAPREDUCE-3299.patch, MAPREDUCE-3299.patch, > MAPREDUCE-3299.patch > > > JobHistory has a table to list all AMs. A similar table can be added to the > AM for info on past failed AMs and the current running one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3326) RM web UI scheduler link not as useful as should be
[ https://issues.apache.org/jira/browse/MAPREDUCE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180732#comment-13180732 ] Arun C Murthy commented on MAPREDUCE-3326: -- Similar to Tom's comment on MAPREDUCE-3299, should we ensure we have the same info via web-services as part of this? > RM web UI scheduler link not as useful as should be > --- > > Key: MAPREDUCE-3326 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3326 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Thomas Graves >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-3326.patch > > > The resource manager web ui page for scheduler doesn't have all the > information about the configuration like the jobtracker page used to have. > The things it seems to show you are the current queues - each queues used, > set, and max percent and then what apps are running in that queue. > It doesn't list any of yarn.scheduler.capacity.maximum-applications, > yarn.scheduler.capacity.maximum-am-resource-percent, > yarn.scheduler.capacity..user-limit-factor, > yarn.scheduler.capacity..minimum-user-limit-percent, queue state, > active users and percent used by user -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3299) Add AMInfo table to the AM job page
[ https://issues.apache.org/jira/browse/MAPREDUCE-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180733#comment-13180733 ] Hadoop QA commented on MAPREDUCE-3299: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509579/MAPREDUCE-3299.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainersMonitor +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1547//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1547//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1547//console This message is automatically generated. > Add AMInfo table to the AM job page > --- > > Key: MAPREDUCE-3299 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3299 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Siddharth Seth >Assignee: Jonathan Eagles >Priority: Minor > Attachments: MAPREDUCE-3299.patch, MAPREDUCE-3299.patch, > MAPREDUCE-3299.patch > > > JobHistory has a table to list all AMs. A similar table can be added to the > AM for info on past failed AMs and the current running one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3299) Add AMInfo table to the AM job page
[ https://issues.apache.org/jira/browse/MAPREDUCE-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180727#comment-13180727 ] Arun C Murthy commented on MAPREDUCE-3299: -- Good point Thomas. Jon, if you don't mind, I'd rather do it as part of this jira... > Add AMInfo table to the AM job page > --- > > Key: MAPREDUCE-3299 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3299 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Siddharth Seth >Assignee: Jonathan Eagles >Priority: Minor > Attachments: MAPREDUCE-3299.patch, MAPREDUCE-3299.patch, > MAPREDUCE-3299.patch > > > JobHistory has a table to list all AMs. A similar table can be added to the > AM for info on past failed AMs and the current running one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3299) Add AMInfo table to the AM job page
[ https://issues.apache.org/jira/browse/MAPREDUCE-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3299: - Target Version/s: 0.23.1 (was: 0.24.0, 0.23.1) Affects Version/s: (was: 0.23.1) (was: 0.24.0) Status: Open (was: Patch Available) > Add AMInfo table to the AM job page > --- > > Key: MAPREDUCE-3299 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3299 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Siddharth Seth >Assignee: Jonathan Eagles >Priority: Minor > Attachments: MAPREDUCE-3299.patch, MAPREDUCE-3299.patch, > MAPREDUCE-3299.patch > > > JobHistory has a table to list all AMs. A similar table can be added to the > AM for info on past failed AMs and the current running one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost nodes in the UI.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180725#comment-13180725 ] Hadoop QA commented on MAPREDUCE-3360: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509577/MAPREDUCE-3360-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1545//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1545//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1545//console This message is automatically generated. > Provide information about lost nodes in the UI. > --- > > Key: MAPREDUCE-3360 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 > Environment: NA >Reporter: Bhallamudi Venkata Siva Kamesh >Assignee: Bhallamudi Venkata Siva Kamesh >Priority: Critical > Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, > MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, > lostNodes.png > > > Currently there is no information provided about *lost nodes*. Provide > information in the UI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3615) mapred ant test failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180722#comment-13180722 ] Hudson commented on MAPREDUCE-3615: --- Integrated in Hadoop-Mapreduce-0.23-Commit #354 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/354/]) merge MAPREDUCE-3615 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java > mapred ant test failures > > > Key: MAPREDUCE-3615 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3615 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3615.patch > > > The following mapred ant tests are failing. This started on December 22nd. > [junit] Running org.apache.hadoop.mapred.TestTrackerBlacklistAcrossJobs > [junit] Running org.apache.hadoop.mapred.TestMiniMRDFSSort > [junit] Running org.apache.hadoop.mapred.TestBadRecords > [junit] Running org.apache.hadoop.mapred.TestClusterMRNotification > [junit] Running org.apache.hadoop.mapred.TestDebugScript > [junit] Running org.apache.hadoop.mapred.TestJobCleanup > [junit] Running org.apache.hadoop.mapred.TestJobClient > [junit] Running org.apache.hadoop.mapred.TestJobHistory > [junit] Running org.apache.hadoop.mapred.TestJobInProgressListener > [junit] Running org.apache.hadoop.mapred.TestJobKillAndFail > [junit] Running org.apache.hadoop.mapred.TestJvmReuse > [junit] Running org.apache.hadoop.mapred.TestKillSubProcesses > [junit] Running org.apache.hadoop.mapred.TestNodeRefresh > [junit] Running org.apache.hadoop.mapred.TestSetupAndCleanupFailure > [junit] Running org.apache.hadoop.mapred.TestTaskFail > [junit] Running org.apache.hadoop.mapred.TestTaskOutputSize > [junit] Running org.apache.hadoop.mapred.TestTaskTrackerSlotManagement > [junit] Running org.apache.hadoop.mapreduce.TestMRJobClient > [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob > [junit] Running > org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-3617: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Patch Available (was: Open) > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-3617: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Open (was: Patch Available) > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
[ https://issues.apache.org/jira/browse/MAPREDUCE-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-3617: --- Attachment: MAPREDUCE-3617.patch > Remove yarn default values for resource manager and nodemanager principal > - > > Key: MAPREDUCE-3617 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1, 0.24.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: MAPREDUCE-3617.patch, MAPREDUCE-3617.patch > > > Default values should be empty since no use can be made of them without > correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3299) Add AMInfo table to the AM job page
[
https://issues.apache.org/jira/browse/MAPREDUCE-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180706#comment-13180706
]
Thomas Graves commented on MAPREDUCE-3299:
--
I think this information should also be made available via the web service call
- something like /mapreduce/jobs/{jobid}/attempts. This is fairly simple and
already implemented in the history server if you look at
/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/AMAttemptInfo.java,
HsJobBlock, and HsWebServices. We could file a separate jira too.
> Add AMInfo table to the AM job page
> ---
>
> Key: MAPREDUCE-3299
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3299
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Affects Versions: 0.23.0, 0.23.1, 0.24.0
>Reporter: Siddharth Seth
>Assignee: Jonathan Eagles
>Priority: Minor
> Attachments: MAPREDUCE-3299.patch, MAPREDUCE-3299.patch,
> MAPREDUCE-3299.patch
>
>
> JobHistory has a table to list all AMs. A similar table can be added to the
> AM for info on past failed AMs and the current running one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3615) mapred ant test failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180700#comment-13180700 ] Hudson commented on MAPREDUCE-3615: --- Integrated in Hadoop-Common-trunk-Commit #1500 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1500/]) MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java > mapred ant test failures > > > Key: MAPREDUCE-3615 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3615 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3615.patch > > > The following mapred ant tests are failing. This started on December 22nd. > [junit] Running org.apache.hadoop.mapred.TestTrackerBlacklistAcrossJobs > [junit] Running org.apache.hadoop.mapred.TestMiniMRDFSSort > [junit] Running org.apache.hadoop.mapred.TestBadRecords > [junit] Running org.apache.hadoop.mapred.TestClusterMRNotification > [junit] Running org.apache.hadoop.mapred.TestDebugScript > [junit] Running org.apache.hadoop.mapred.TestJobCleanup > [junit] Running org.apache.hadoop.mapred.TestJobClient > [junit] Running org.apache.hadoop.mapred.TestJobHistory > [junit] Running org.apache.hadoop.mapred.TestJobInProgressListener > [junit] Running org.apache.hadoop.mapred.TestJobKillAndFail > [junit] Running org.apache.hadoop.mapred.TestJvmReuse > [junit] Running org.apache.hadoop.mapred.TestKillSubProcesses > [junit] Running org.apache.hadoop.mapred.TestNodeRefresh > [junit] Running org.apache.hadoop.mapred.TestSetupAndCleanupFailure > [junit] Running org.apache.hadoop.mapred.TestTaskFail > [junit] Running org.apache.hadoop.mapred.TestTaskOutputSize > [junit] Running org.apache.hadoop.mapred.TestTaskTrackerSlotManagement > [junit] Running org.apache.hadoop.mapreduce.TestMRJobClient > [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob > [junit] Running > org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3615) mapred ant test failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180696#comment-13180696 ] Hudson commented on MAPREDUCE-3615: --- Integrated in Hadoop-Common-0.23-Commit #343 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/343/]) merge MAPREDUCE-3615 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java > mapred ant test failures > > > Key: MAPREDUCE-3615 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3615 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3615.patch > > > The following mapred ant tests are failing. This started on December 22nd. > [junit] Running org.apache.hadoop.mapred.TestTrackerBlacklistAcrossJobs > [junit] Running org.apache.hadoop.mapred.TestMiniMRDFSSort > [junit] Running org.apache.hadoop.mapred.TestBadRecords > [junit] Running org.apache.hadoop.mapred.TestClusterMRNotification > [junit] Running org.apache.hadoop.mapred.TestDebugScript > [junit] Running org.apache.hadoop.mapred.TestJobCleanup > [junit] Running org.apache.hadoop.mapred.TestJobClient > [junit] Running org.apache.hadoop.mapred.TestJobHistory > [junit] Running org.apache.hadoop.mapred.TestJobInProgressListener > [junit] Running org.apache.hadoop.mapred.TestJobKillAndFail > [junit] Running org.apache.hadoop.mapred.TestJvmReuse > [junit] Running org.apache.hadoop.mapred.TestKillSubProcesses > [junit] Running org.apache.hadoop.mapred.TestNodeRefresh > [junit] Running org.apache.hadoop.mapred.TestSetupAndCleanupFailure > [junit] Running org.apache.hadoop.mapred.TestTaskFail > [junit] Running org.apache.hadoop.mapred.TestTaskOutputSize > [junit] Running org.apache.hadoop.mapred.TestTaskTrackerSlotManagement > [junit] Running org.apache.hadoop.mapreduce.TestMRJobClient > [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob > [junit] Running > org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3615) mapred ant test failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180694#comment-13180694 ] Hudson commented on MAPREDUCE-3615: --- Integrated in Hadoop-Hdfs-trunk-Commit #1572 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1572/]) MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java > mapred ant test failures > > > Key: MAPREDUCE-3615 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3615 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3615.patch > > > The following mapred ant tests are failing. This started on December 22nd. > [junit] Running org.apache.hadoop.mapred.TestTrackerBlacklistAcrossJobs > [junit] Running org.apache.hadoop.mapred.TestMiniMRDFSSort > [junit] Running org.apache.hadoop.mapred.TestBadRecords > [junit] Running org.apache.hadoop.mapred.TestClusterMRNotification > [junit] Running org.apache.hadoop.mapred.TestDebugScript > [junit] Running org.apache.hadoop.mapred.TestJobCleanup > [junit] Running org.apache.hadoop.mapred.TestJobClient > [junit] Running org.apache.hadoop.mapred.TestJobHistory > [junit] Running org.apache.hadoop.mapred.TestJobInProgressListener > [junit] Running org.apache.hadoop.mapred.TestJobKillAndFail > [junit] Running org.apache.hadoop.mapred.TestJvmReuse > [junit] Running org.apache.hadoop.mapred.TestKillSubProcesses > [junit] Running org.apache.hadoop.mapred.TestNodeRefresh > [junit] Running org.apache.hadoop.mapred.TestSetupAndCleanupFailure > [junit] Running org.apache.hadoop.mapred.TestTaskFail > [junit] Running org.apache.hadoop.mapred.TestTaskOutputSize > [junit] Running org.apache.hadoop.mapred.TestTaskTrackerSlotManagement > [junit] Running org.apache.hadoop.mapreduce.TestMRJobClient > [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob > [junit] Running > org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3622) findbug error during test-patch: org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent defines equals but not hashCode
[ https://issues.apache.org/jira/browse/MAPREDUCE-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180690#comment-13180690 ] Robert Joseph Evans commented on MAPREDUCE-3622: Strike that last comment. I guess eclipse some how got confused and pointed me to the wrong file. My first comment still stands. As part of MAPREDUCE-3312 I will be fixing this issue because the need for the equals hack will not longer be needed. I should upload a patch to MAPREDUCE-3312 shortly. > findbug error during test-patch: > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent > defines equals but not hashCode > --- > > Key: MAPREDUCE-3622 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3622 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves > > findbug error: > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent > defines equals but not hashCode > first time I see this is build 1540: > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1540//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
