[jira] [Updated] (MAPREDUCE-2562) NullPointerException in Jobtracker when it is started without Name Node
[
https://issues.apache.org/jira/browse/MAPREDUCE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-2562:
-
Affects Version/s: 0.22.0
Fix Version/s: 0.22.0
This issue exist in 0.22 also. Updated the versions.
NullPointerException in Jobtracker when it is started without Name Node
---
Key: MAPREDUCE-2562
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2562
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.22.0, 0.23.0
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 0.22.0, 0.23.0
Attachments: MAPREDUCE-2562.patch
It is throwing NullPointerException in job tracker logs when job tracker is
started without NameNode.
{code:title=Bar.java|borderStyle=solid}
2011-06-03 01:50:04,304 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.18.52.225:9000. Already tried 7 time(s).
2011-06-03 01:50:05,307 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.18.52.225:9000. Already tried 8 time(s).
2011-06-03 01:50:06,310 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.18.52.225:9000. Already tried 9 time(s).
2011-06-03 01:50:21,243 FATAL org.apache.hadoop.mapred.JobTracker:
java.lang.NullPointerException
at org.apache.hadoop.mapred.JobTracker.init(JobTracker.java:1635)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:287)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:279)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:274)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4312)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2777) Backport MAPREDUCE-220 to Hadoop 20 security branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat updated MAPREDUCE-2777: -- Attachment: mapreduce-2777-v1.3.patch Attaching a patch that backports the MAPREDUCE-220 and MAPREDUCE-2469 features to the hadoop-0.20-security branch. MAPREDUCE-220 and MAPREDUCE-2469 have dependencies which also got selectively backported. Here is the list of JIRA's that got backported: MR-220, MR-1201, MR-1762, MR-1218 MR-2469. I have resolved most of the hunks w.r.t parameter naming, missing APIs and missing backports (e.g GC_TIME_MILLIS counters etc). We have tested this patch at scale and also benchmarked this patch for regressions. It seems that the overhead is negligible and the newly added counters are correctly captured and recorded. Backport MAPREDUCE-220 to Hadoop 20 security branch --- Key: MAPREDUCE-2777 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2777 Project: Hadoop Map/Reduce Issue Type: New Feature Affects Versions: 0.20.205.0 Reporter: Jonathan Eagles Assignee: Amar Kamat Attachments: mapreduce-2777-v1.3.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2562) NullPointerException in Jobtracker when it is started without Name Node
[
https://issues.apache.org/jira/browse/MAPREDUCE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097026#comment-13097026
]
Harsh J commented on MAPREDUCE-2562:
Awesome, thanks for testing and updating the fields! :)
NullPointerException in Jobtracker when it is started without Name Node
---
Key: MAPREDUCE-2562
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2562
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.22.0, 0.23.0
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 0.22.0, 0.23.0
Attachments: MAPREDUCE-2562.patch
It is throwing NullPointerException in job tracker logs when job tracker is
started without NameNode.
{code:title=Bar.java|borderStyle=solid}
2011-06-03 01:50:04,304 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.18.52.225:9000. Already tried 7 time(s).
2011-06-03 01:50:05,307 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.18.52.225:9000. Already tried 8 time(s).
2011-06-03 01:50:06,310 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.18.52.225:9000. Already tried 9 time(s).
2011-06-03 01:50:21,243 FATAL org.apache.hadoop.mapred.JobTracker:
java.lang.NullPointerException
at org.apache.hadoop.mapred.JobTracker.init(JobTracker.java:1635)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:287)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:279)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:274)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4312)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2907) ResourceManager logs filled with [INFO] debug messages from org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue
[
https://issues.apache.org/jira/browse/MAPREDUCE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated MAPREDUCE-2907:
Attachment: MAPREDUCE-2907.3.patch
Removing messages from LeafQueue and CapacityScheduler too.
There are constant heartbeat LOG messages like
{noformat}2011-09-05 03:02:58,207 INFO capacity.CapacityScheduler
(CapacityScheduler.java:nodeUpdate(513)) - nodeUpdate: 127.0.0.1:45454
clusterResources: memory: 8192,
Call:
protocol=org.apache.hadoop.yarn.proto.ResourceTracker.ResourceTrackerService.BlockingInterface,
method=nodeHeartbeat
2011-09-05 03:02:59,209 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(280)) -
Processing 127.0.0.1:45454 of type STATUS_UPDATE
{noformat}
Should we disable them as well?
ResourceManager logs filled with [INFO] debug messages from
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue
Key: MAPREDUCE-2907
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2907
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.0
Reporter: Ravi Prakash
Assignee: Ravi Prakash
Fix For: 0.23.0
Attachments: MAPREDUCE-2907.1.patch, MAPREDUCE-2907.2.patch,
MAPREDUCE-2907.3.patch
I see a lot of info messages (probably used for debugging during development)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2697: - Attachment: MAPREDUCE-2697.patch Updated patch, including unit tests. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2697: - Status: Patch Available (was: Open) Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097087#comment-13097087 ] Hadoop QA commented on MAPREDUCE-2697: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493013/MAPREDUCE-2697.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestParentQueue +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/606//console This message is automatically generated. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2846) a small % of all tasks fail with DefaultTaskController
[ https://issues.apache.org/jira/browse/MAPREDUCE-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097158#comment-13097158 ] Devaraj K commented on MAPREDUCE-2846: -- I know it is too late to comment here but just want to inform. The trunk patch is not required because writeToIndexFile() which is private and calling from only one place i.e syncLogs() which is already synchronized method. a small % of all tasks fail with DefaultTaskController -- Key: MAPREDUCE-2846 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2846 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, task-controller, tasktracker Affects Versions: 0.20.204.0 Reporter: Allen Wittenauer Assignee: Owen O'Malley Priority: Blocker Fix For: 0.20.204.0, 0.23.0 Attachments: sync-trunk.patch, sync.patch After upgrading our test 0.20.203 grid to 0.20.204-rc2, we ran terasort to verify operation. While the job completed successfully, approx 10% of the tasks failed with task runner execution errors and the inability to create symlinks for attempt logs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2926) 500 Error in ResourceManager UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Joseph Evans updated MAPREDUCE-2926:
---
Status: Open (was: Patch Available)
Canceling patch because it is a duplicate of HADOOP-7606. It looks like the
500 error is still there in the latest code, so this JIRA can be used for that.
500 Error in ResourceManager UI
---
Key: MAPREDUCE-2926
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2926
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0, 0.24.0
Reporter: Robert Joseph Evans
Assignee: Robert Joseph Evans
Fix For: 0.23.0, 0.24.0
Attachments: MR-2926-v1.patch, MR-2926-v2.patch
When accessing the resource manager UI the following is returned
{noformat}
Problem accessing /. Reason:
org.codehaus.jackson.type.JavaType.init(Ljava/lang/Class;)V
Caused by:
java.lang.NoSuchMethodError:
org.codehaus.jackson.type.JavaType.init(Ljava/lang/Class;)V
at org.codehaus.jackson.map.type.TypeBase.init(TypeBase.java:15)
at org.codehaus.jackson.map.type.SimpleType.init(SimpleType.java:45)
at org.codehaus.jackson.map.type.SimpleType.init(SimpleType.java:40)
at
org.codehaus.jackson.map.type.TypeBindings.clinit(TypeBindings.java:20)
at
org.codehaus.jackson.map.type.TypeFactory._fromType(TypeFactory.java:530)
at org.codehaus.jackson.map.type.TypeFactory.type(TypeFactory.java:63)
at org.codehaus.jackson.map.ObjectMapper.clinit(ObjectMapper.java:179)
at org.apache.hadoop.yarn.webapp.Controller.clinit(Controller.java:43)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
com.google.inject.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:81)
at
com.google.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
at
com.google.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:111)
at com.google.inject.InjectorImpl$4$1.call(InjectorImpl.java:758)
at com.google.inject.InjectorImpl.callInContext(InjectorImpl.java:804)
at com.google.inject.InjectorImpl$4.get(InjectorImpl.java:754)
at com.google.inject.InjectorImpl.getInstance(InjectorImpl.java:793)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:136)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:216)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:141)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:63)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:892)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
[jira] [Created] (MAPREDUCE-2932) Missing instrumentation plugin class shouldn't crash the TT startup per design
Missing instrumentation plugin class shouldn't crash the TT startup per design
--
Key: MAPREDUCE-2932
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2932
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 0.20.2
Reporter: Harsh J
Assignee: Harsh J
Priority: Trivial
Per the implementation of the TaskTracker instrumentation plugin implementation
(from 2008), a ClassNotFoundException during loading up of an configured
TaskTracker instrumentation class shouldn't have hampered TT start up at all.
But, there is one class-fetching call outside try/catch, which makes TT fall
down with a RuntimeException if there's a class not found. Would be good to
include this line into the try/catch itself.
Strace would appear as:
{code}
2011-08-25 11:45:38,470 ERROR org.apache.hadoop.mapred.TaskTracker: Can not
start task tracker because java.lang.RuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.mapred.CustomInstPlugin
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java)
at
org.apache.hadoop.mapred.TaskTracker.getInstrumentationClass(TaskTracker.java)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2932) Missing instrumentation plugin class shouldn't crash the TT startup per design
[
https://issues.apache.org/jira/browse/MAPREDUCE-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2932:
---
Attachment: 0.20-MAPREDUCE-2932.r1.diff
Patch for 0.20-security. Unsure if required for 0.22 and for any of new
components of 0.23+, will have to investigate.
Missing instrumentation plugin class shouldn't crash the TT startup per design
--
Key: MAPREDUCE-2932
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2932
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 0.20.2
Reporter: Harsh J
Assignee: Harsh J
Priority: Trivial
Attachments: 0.20-MAPREDUCE-2932.r1.diff
Per the implementation of the TaskTracker instrumentation plugin
implementation (from 2008), a ClassNotFoundException during loading up of an
configured TaskTracker instrumentation class shouldn't have hampered TT start
up at all.
But, there is one class-fetching call outside try/catch, which makes TT fall
down with a RuntimeException if there's a class not found. Would be good to
include this line into the try/catch itself.
Strace would appear as:
{code}
2011-08-25 11:45:38,470 ERROR org.apache.hadoop.mapred.TaskTracker: Can not
start task tracker because java.lang.RuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.mapred.CustomInstPlugin
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java)
at
org.apache.hadoop.mapred.TaskTracker.getInstrumentationClass(TaskTracker.java)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2932) Missing instrumentation plugin class shouldn't crash the TT startup per design
[
https://issues.apache.org/jira/browse/MAPREDUCE-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2932:
---
Status: Patch Available (was: Open)
Missing instrumentation plugin class shouldn't crash the TT startup per design
--
Key: MAPREDUCE-2932
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2932
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 0.20.2
Reporter: Harsh J
Assignee: Harsh J
Priority: Trivial
Attachments: 0.20-MAPREDUCE-2932.r1.diff
Per the implementation of the TaskTracker instrumentation plugin
implementation (from 2008), a ClassNotFoundException during loading up of an
configured TaskTracker instrumentation class shouldn't have hampered TT start
up at all.
But, there is one class-fetching call outside try/catch, which makes TT fall
down with a RuntimeException if there's a class not found. Would be good to
include this line into the try/catch itself.
Strace would appear as:
{code}
2011-08-25 11:45:38,470 ERROR org.apache.hadoop.mapred.TaskTracker: Can not
start task tracker because java.lang.RuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.mapred.CustomInstPlugin
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java)
at
org.apache.hadoop.mapred.TaskTracker.getInstrumentationClass(TaskTracker.java)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2932) Missing instrumentation plugin class shouldn't crash the TT startup per design
[
https://issues.apache.org/jira/browse/MAPREDUCE-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097198#comment-13097198
]
Hadoop QA commented on MAPREDUCE-2932:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12493041/0.20-MAPREDUCE-2932.r1.diff
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 patch. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/607//console
This message is automatically generated.
Missing instrumentation plugin class shouldn't crash the TT startup per design
--
Key: MAPREDUCE-2932
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2932
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 0.20.2
Reporter: Harsh J
Assignee: Harsh J
Priority: Trivial
Attachments: 0.20-MAPREDUCE-2932.r1.diff
Per the implementation of the TaskTracker instrumentation plugin
implementation (from 2008), a ClassNotFoundException during loading up of an
configured TaskTracker instrumentation class shouldn't have hampered TT start
up at all.
But, there is one class-fetching call outside try/catch, which makes TT fall
down with a RuntimeException if there's a class not found. Would be good to
include this line into the try/catch itself.
Strace would appear as:
{code}
2011-08-25 11:45:38,470 ERROR org.apache.hadoop.mapred.TaskTracker: Can not
start task tracker because java.lang.RuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.mapred.CustomInstPlugin
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java)
at
org.apache.hadoop.mapred.TaskTracker.getInstrumentationClass(TaskTracker.java)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2584) Check for serializers early, and give out more information regarding missing serializers
[ https://issues.apache.org/jira/browse/MAPREDUCE-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-2584: --- Attachment: MAPREDUCE-2584.r5.diff Updating patch for trunk… Check for serializers early, and give out more information regarding missing serializers Key: MAPREDUCE-2584 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2584 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.20.2 Reporter: Harsh J Assignee: Harsh J Labels: serializers, tasks Fix For: 0.23.0 Attachments: MAPREDUCE-2584.r2.diff, MAPREDUCE-2584.r3.diff, MAPREDUCE-2584.r4.diff, MAPREDUCE-2584.r5.diff As discussed on HADOOP-7328, MapReduce can handle serializers in a much better way in case of bad configuration, improper imports (Some odd Text class instead of the Writable Text set as key), etc.. This issue covers the MapReduce parts of the improvements (made to IFile, MapOutputBuffer, etc. and possible early-check of serializer availability pre-submit) that provide more information than just an NPE as is the current case. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2584) Check for serializers early, and give out more information regarding missing serializers
[ https://issues.apache.org/jira/browse/MAPREDUCE-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097211#comment-13097211 ] Hadoop QA commented on MAPREDUCE-2584: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493044/MAPREDUCE-2584.r5.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/608//console This message is automatically generated. Check for serializers early, and give out more information regarding missing serializers Key: MAPREDUCE-2584 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2584 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.20.2 Reporter: Harsh J Assignee: Harsh J Labels: serializers, tasks Fix For: 0.23.0 Attachments: MAPREDUCE-2584.r2.diff, MAPREDUCE-2584.r3.diff, MAPREDUCE-2584.r4.diff, MAPREDUCE-2584.r5.diff As discussed on HADOOP-7328, MapReduce can handle serializers in a much better way in case of bad configuration, improper imports (Some odd Text class instead of the Writable Text set as key), etc.. This issue covers the MapReduce parts of the improvements (made to IFile, MapOutputBuffer, etc. and possible early-check of serializer availability pre-submit) that provide more information than just an NPE as is the current case. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2697: - Status: Open (was: Patch Available) Checking the test failures... Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2191) Findbugs reports 13 warnings on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy resolved MAPREDUCE-2191. -- Resolution: Duplicate Duplicate of MAPREDUCE-2193 (which has a patch, albeit stale). Findbugs reports 13 warnings on trunk - Key: MAPREDUCE-2191 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2191 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 0.22.0 Reporter: Eli Collins Attachments: hadoop-findbugs-report.html Findbugs reports 13 warnings on trunk: Warning Type Number Bad practice Warnings 1 Correctness Warnings 5 Multithreaded correctness Warnings6 Performance Warnings 1 Total 13 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2697: - Attachment: MAPREDUCE-2697.patch Fixed failures caused due to low limits on activated apps. Findbugs warnings seem unrelated - tracked in MAPREDUCE-2908. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2697: - Status: Patch Available (was: Open) Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2191) Findbugs reports 13 warnings on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097255#comment-13097255 ] Harsh J commented on MAPREDUCE-2191: Yup I was targeting at 0.22. The trunk refactoring has taken it outta shape. Will renew on MAPREDUCE-2193 shortly, but looks like findbugs reports need some work locally first, doing that :) Findbugs reports 13 warnings on trunk - Key: MAPREDUCE-2191 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2191 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 0.22.0 Reporter: Eli Collins Attachments: hadoop-findbugs-report.html Findbugs reports 13 warnings on trunk: Warning Type Number Bad practice Warnings 1 Correctness Warnings 5 Multithreaded correctness Warnings6 Performance Warnings 1 Total 13 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097260#comment-13097260 ] Hadoop QA commented on MAPREDUCE-2697: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493055/MAPREDUCE-2697.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/609//console This message is automatically generated. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097284#comment-13097284 ] Arun C Murthy commented on MAPREDUCE-2697: -- Forgot to add - the crux of the solution is that we limit the amount of 'overhead' for applications in the cluster to a percent defined by yarn.capacity-scheduler.maximum-am-resource-percent. Thus, the #concurrent apps scales well with cluster capacity and #apps don't have to 'configured' by admins. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2697: - Attachment: MAPREDUCE-2697.patch Updated patch to include new config in capacity-scheduler.xml, no code changes. Also, default value of yarn.capacity-scheduler.maximum-am-resource-percent is 0.1, thus no more than 10% of the cluster is taken up by AMs and hence caps the #concurrent-apps. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097293#comment-13097293 ] Hadoop QA commented on MAPREDUCE-2697: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493062/MAPREDUCE-2697.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/610//console This message is automatically generated. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097290#comment-13097290 ] Hudson commented on MAPREDUCE-2697: --- Integrated in Hadoop-Common-trunk-Commit #830 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/830/]) MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running applications per-queue per-user. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1165403 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ContainerIdPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Queue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/resources/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097294#comment-13097294 ] Hudson commented on MAPREDUCE-2697: --- Integrated in Hadoop-Hdfs-trunk-Commit #907 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/907/]) MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running applications per-queue per-user. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1165403 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ContainerIdPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Queue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/resources/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2697: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097298#comment-13097298 ] Hudson commented on MAPREDUCE-2697: --- Integrated in Hadoop-Mapreduce-trunk-Commit #840 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/840/]) MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running applications per-queue per-user. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1165403 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ContainerIdPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Queue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/resources/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java Enhance CS to cap concurrently running jobs --- Key: MAPREDUCE-2697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.0 Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, MAPREDUCE-2697.patch Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097608#comment-13097608 ] Arun C Murthy commented on MAPREDUCE-2882: -- +1 TestLineRecordReader depends on ant jars Key: MAPREDUCE-2882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.23.0 Attachments: mr-2882.txt This test is currently importing an ant utility class to read a file - this dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2933) Change allocate call to return ContainerStatus for completed containers rather than Container
Change allocate call to return ContainerStatus for completed containers rather than Container -- Key: MAPREDUCE-2933 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2933 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager, resourcemanager Reporter: Arun C Murthy Assignee: Arun C Murthy Change allocate call to return ContainerStatus for completed containers rather than Container, we should do this all the way from the NodeManager too. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2933) Change allocate call to return ContainerStatus for completed containers rather than Container
[ https://issues.apache.org/jira/browse/MAPREDUCE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2933: - Priority: Blocker (was: Major) Change allocate call to return ContainerStatus for completed containers rather than Container -- Key: MAPREDUCE-2933 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2933 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: applicationmaster, mrv2, nodemanager, resourcemanager Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.0 Change allocate call to return ContainerStatus for completed containers rather than Container, we should do this all the way from the NodeManager too. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2895) Merge AllocateResponse and AMResponse
[ https://issues.apache.org/jira/browse/MAPREDUCE-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy resolved MAPREDUCE-2895. -- Resolution: Won't Fix The reason we want AMResponse is to be able to store it to track in on the RM, thus we need to keep them separate. Merge AllocateResponse and AMResponse - Key: MAPREDUCE-2895 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2895 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Arun C Murthy Fix For: 0.23.0 We need to merge AllocateResponse and AMResponse, having them separate serves no purpose. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2900) Replace ContainerId, Resource in ContainerLaunchContext with Container
[ https://issues.apache.org/jira/browse/MAPREDUCE-2900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097624#comment-13097624 ] Arun C Murthy commented on MAPREDUCE-2900: -- Vinod, should we close this as a wontfix? Replace ContainerId, Resource in ContainerLaunchContext with Container -- Key: MAPREDUCE-2900 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2900 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Fix For: 0.23.0 Replace ContainerId, Resource in ContainerLaunchContext with Container -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2900) Replace ContainerId, Resource in ContainerLaunchContext with Container
[ https://issues.apache.org/jira/browse/MAPREDUCE-2900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy resolved MAPREDUCE-2900. -- Resolution: Won't Fix Wontfix per Vinod's comment. Replace ContainerId, Resource in ContainerLaunchContext with Container -- Key: MAPREDUCE-2900 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2900 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Fix For: 0.23.0 Replace ContainerId, Resource in ContainerLaunchContext with Container -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097628#comment-13097628 ] Hudson commented on MAPREDUCE-2882: --- Integrated in Hadoop-Hdfs-trunk-Commit #908 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/908/]) MAPREDUCE-2882. TestLineRecordReader depends on ant jars. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1165449 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLineRecordReader.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/lib/input/TestLineRecordReader.java TestLineRecordReader depends on ant jars Key: MAPREDUCE-2882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.23.0 Attachments: mr-2882.txt This test is currently importing an ant utility class to read a file - this dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097631#comment-13097631 ] Hudson commented on MAPREDUCE-2882: --- Integrated in Hadoop-Mapreduce-trunk-Commit #841 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/841/]) MAPREDUCE-2882. TestLineRecordReader depends on ant jars. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1165449 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLineRecordReader.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/lib/input/TestLineRecordReader.java TestLineRecordReader depends on ant jars Key: MAPREDUCE-2882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.23.0 Attachments: mr-2882.txt This test is currently importing an ant utility class to read a file - this dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2690) Construct the web page for default scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-2690: -- Attachment: MAPREDUCE-2690.2011-09-05.patch Fixed findbugs warning about wrote to static field from instance method. The other findbugs warnings were already there. Construct the web page for default scheduler Key: MAPREDUCE-2690 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2690 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Eric Payne Fix For: 0.23.0 Attachments: DefaultSchedulerPage.jpg, MAPREDUCE-2690.2011-09-02.patch, MAPREDUCE-2690.2011-09-05.patch Currently, the web page for default scheduler reads as Under construction. This is a long known issue, but could not find a tracking ticket. Hence opening one. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097634#comment-13097634 ] Arun C Murthy commented on MAPREDUCE-2882: -- Todd, shouldn't we commit this to 0.23 also? TestLineRecordReader depends on ant jars Key: MAPREDUCE-2882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.23.0 Attachments: mr-2882.txt This test is currently importing an ant utility class to read a file - this dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2690) Construct the web page for default scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097641#comment-13097641 ] Hadoop QA commented on MAPREDUCE-2690: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493084/MAPREDUCE-2690.2011-09-05.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in .. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/611//console This message is automatically generated. Construct the web page for default scheduler Key: MAPREDUCE-2690 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2690 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Eric Payne Fix For: 0.23.0 Attachments: DefaultSchedulerPage.jpg, MAPREDUCE-2690.2011-09-02.patch, MAPREDUCE-2690.2011-09-05.patch Currently, the web page for default scheduler reads as Under construction. This is a long known issue, but could not find a tracking ticket. Hence opening one. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097659#comment-13097659 ] Todd Lipcon commented on MAPREDUCE-2882: Yes, I did commit this to both trunk and 23. I think the integration build for 23 only runs nightly, hence no comment yet. TestLineRecordReader depends on ant jars Key: MAPREDUCE-2882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.23.0 Attachments: mr-2882.txt This test is currently importing an ant utility class to read a file - this dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2934) MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
--
Key: MAPREDUCE-2934
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2934
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.24.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Fix For: 0.24.0
Once HADOOP-7607 goes in, {{ProtoOverHadoopRpcEngine.stopProxy}} will need to
be removed or at least have its {{@Override}} annotation removed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2934) MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
[
https://issues.apache.org/jira/browse/MAPREDUCE-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated MAPREDUCE-2934:
--
Attachment: mapreduce-2934.0.patch
Patch which addresses the issue.
MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
--
Key: MAPREDUCE-2934
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2934
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.24.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Fix For: 0.24.0
Attachments: mapreduce-2934.0.patch
Once HADOOP-7607 goes in, {{ProtoOverHadoopRpcEngine.stopProxy}} will need to
be removed or at least have its {{@Override}} annotation removed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch apps in both secure and non-secure cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated MAPREDUCE-2687:
-
Attachment: MAPREDUCE-2687.patch
With this updated patch I can run applications as other users than the mapred
user.
OTOH, there is a different bug which causes apps from other users to fail due
to an error in LTC:
{noformat}
2011-09-06 02:29:07,379 WARN
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception
from container-launch :
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
at org.apache.hadoop.util.Shell.run(Shell.java:188)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:171)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:191)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:61)
{noformat}
Still tracking this down... I wonder if this causes permission issues.
I've committed HADOOP-7580 which should help ease debugging.
Non superusers unable to launch apps in both secure and non-secure cluster
--
Key: MAPREDUCE-2687
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Assignee: Mahadev konar
Priority: Blocker
Fix For: 0.23.0
Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
Apps of non superuser fail to succeed in both secure and non-secure
environment. Only the superuser(i.e. one who started/owns the mrv2 cluster)
is able to launch apps successfully. However, when a normal user submits a
job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch apps in both secure and non-secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097716#comment-13097716 ] Arun C Murthy commented on MAPREDUCE-2687: -- 2011-09-06 02:29:07,378 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 1 Non superusers unable to launch apps in both secure and non-secure cluster -- Key: MAPREDUCE-2687 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1720) 'Killed' jobs and 'Failed' jobs should be displayed seperately in JobTracker UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated MAPREDUCE-1720: --- Fix Version/s: (was: 0.23.0) 0.22.0 0.20.205.0 I can provide an updated patch for 0.20-security branch if there's still interest in this. 'Killed' jobs and 'Failed' jobs should be displayed seperately in JobTracker UI Key: MAPREDUCE-1720 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1720 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.1 Environment: all Reporter: Subramaniam Krishnan Assignee: Harsh J Fix For: 0.20.205.0, 0.22.0 Attachments: mapred.failed.killed.difference.png, mapreduce.unsuccessfuljobs.ui.r1.diff The JobTracker UI shows both Failed/Killed Jobs as Failed. The Killed job status has been separated from Failed as part of HADOOP-3924, so the UI needs to be updated to reflect the same. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2784) [Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat updated MAPREDUCE-2784: -- Status: Patch Available (was: Open) [Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode. --- Key: MAPREDUCE-2784 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2784 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/gridmix Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, junit Attachments: mapreduce-2784-v1.3.patch, mapreduce-2784-v1.4.patch, mapreduce-2784-v1.5.patch TestGridmixSummary fails with NPE when run in debug mode. JobFactory tries to access the _createReaderThread()_ API of JobStoryProducer which returns null in TestGridmixSummary's FakeJobStoryProducer. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2784) [Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat updated MAPREDUCE-2784: -- Attachment: mapreduce-2784-v1.5.patch Hadoop source folder names changed. Updating and re-uploading the earlier patch to apply cleanly. [Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode. --- Key: MAPREDUCE-2784 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2784 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/gridmix Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, junit Attachments: mapreduce-2784-v1.3.patch, mapreduce-2784-v1.4.patch, mapreduce-2784-v1.5.patch TestGridmixSummary fails with NPE when run in debug mode. JobFactory tries to access the _createReaderThread()_ API of JobStoryProducer which returns null in TestGridmixSummary's FakeJobStoryProducer. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2784) [Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097754#comment-13097754 ] Hadoop QA commented on MAPREDUCE-2784: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493116/mapreduce-2784-v1.5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/612//console This message is automatically generated. [Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode. --- Key: MAPREDUCE-2784 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2784 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/gridmix Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, junit Attachments: mapreduce-2784-v1.3.patch, mapreduce-2784-v1.4.patch, mapreduce-2784-v1.5.patch TestGridmixSummary fails with NPE when run in debug mode. JobFactory tries to access the _createReaderThread()_ API of JobStoryProducer which returns null in TestGridmixSummary's FakeJobStoryProducer. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
