[jira] [Created] (YARN-2013) The diagnostics is always the ExitCodeException stack when the container crashes
Zhijie Shen created YARN-2013:
-
Summary: The diagnostics is always the ExitCodeException stack
when the container crashes
Key: YARN-2013
URL: https://issues.apache.org/jira/browse/YARN-2013
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Reporter: Zhijie Shen
When a container crashes, ExitCodeException will be thrown from Shell.
Default/LinuxContainerExecutor captures the exception, put the exception stack
into the diagnostic. Therefore, the exception stack is always the same.
{code}
String diagnostics = Exception from container-launch: \n
+ StringUtils.stringifyException(e) + \n + shExec.getOutput();
container.handle(new ContainerDiagnosticsUpdateEvent(containerId,
diagnostics));
{code}
In addition, it seems that the exception always has a empty message as there's
no message from stderr. Hence the diagnostics is not of much use for users to
analyze the reason of container crash.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2013) The diagnostics is always the ExitCodeException stack when the container crashes
[
https://issues.apache.org/jira/browse/YARN-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986560#comment-13986560
]
Tsuyoshi OZAWA commented on YARN-2013:
--
+1(non-binding), in fact I often faces the problem in Tez project. This
behavior make it difficult to debug YARN Apps.
TestContainerLaunch#testContainerLaunchStdoutAndStderrDiagnostics() confirms
that Exception.getMessages() returns the stderr's message. We should include
the message into the diagnostics.
The diagnostics is always the ExitCodeException stack when the container
crashes
Key: YARN-2013
URL: https://issues.apache.org/jira/browse/YARN-2013
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Reporter: Zhijie Shen
When a container crashes, ExitCodeException will be thrown from Shell.
Default/LinuxContainerExecutor captures the exception, put the exception
stack into the diagnostic. Therefore, the exception stack is always the same.
{code}
String diagnostics = Exception from container-launch: \n
+ StringUtils.stringifyException(e) + \n + shExec.getOutput();
container.handle(new ContainerDiagnosticsUpdateEvent(containerId,
diagnostics));
{code}
In addition, it seems that the exception always has a empty message as
there's no message from stderr. Hence the diagnostics is not of much use for
users to analyze the reason of container crash.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (YARN-2013) The diagnostics is always the ExitCodeException stack when the container crashes
[
https://issues.apache.org/jira/browse/YARN-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi OZAWA reassigned YARN-2013:
Assignee: Tsuyoshi OZAWA
The diagnostics is always the ExitCodeException stack when the container
crashes
Key: YARN-2013
URL: https://issues.apache.org/jira/browse/YARN-2013
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Reporter: Zhijie Shen
Assignee: Tsuyoshi OZAWA
When a container crashes, ExitCodeException will be thrown from Shell.
Default/LinuxContainerExecutor captures the exception, put the exception
stack into the diagnostic. Therefore, the exception stack is always the same.
{code}
String diagnostics = Exception from container-launch: \n
+ StringUtils.stringifyException(e) + \n + shExec.getOutput();
container.handle(new ContainerDiagnosticsUpdateEvent(containerId,
diagnostics));
{code}
In addition, it seems that the exception always has a empty message as
there's no message from stderr. Hence the diagnostics is not of much use for
users to analyze the reason of container crash.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2013) The diagnostics is always the ExitCodeException stack when the container crashes
[
https://issues.apache.org/jira/browse/YARN-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi OZAWA updated YARN-2013:
-
Attachment: YARN-2013.1.patch
Changed to include stderr output in the diagnostics.
The diagnostics is always the ExitCodeException stack when the container
crashes
Key: YARN-2013
URL: https://issues.apache.org/jira/browse/YARN-2013
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Reporter: Zhijie Shen
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2013.1.patch
When a container crashes, ExitCodeException will be thrown from Shell.
Default/LinuxContainerExecutor captures the exception, put the exception
stack into the diagnostic. Therefore, the exception stack is always the same.
{code}
String diagnostics = Exception from container-launch: \n
+ StringUtils.stringifyException(e) + \n + shExec.getOutput();
container.handle(new ContainerDiagnosticsUpdateEvent(containerId,
diagnostics));
{code}
In addition, it seems that the exception always has a empty message as
there's no message from stderr. Hence the diagnostics is not of much use for
users to analyze the reason of container crash.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1342) Recover container tokens upon nodemanager restart
[
https://issues.apache.org/jira/browse/YARN-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986605#comment-13986605
]
Hadoop QA commented on YARN-1342:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12642738/YARN-1342v3-and-YARN-1987.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3670//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3670//console
This message is automatically generated.
Recover container tokens upon nodemanager restart
-
Key: YARN-1342
URL: https://issues.apache.org/jira/browse/YARN-1342
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Affects Versions: 2.3.0
Reporter: Jason Lowe
Assignee: Jason Lowe
Attachments: YARN-1342.patch, YARN-1342v2.patch,
YARN-1342v3-and-YARN-1987.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-1201:
-
Issue Type: Bug (was: Test)
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986644#comment-13986644
]
Junping Du commented on YARN-1201:
--
Thanks [~leftnoteasy] for identifying the root cause and delivering a patch
here. However, I don't think we should fix test here as test code itself is
fine to me. We'd better to throw an exception with meaningful message rather
than NPE when local hostname cannot be resolved. So I move this from test to
bug.
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-1339) Recover DeletionService state upon nodemanager restart
[ https://issues.apache.org/jira/browse/YARN-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1339: - Attachment: YARN-1339v3-and-YARN-1987.patch Updating the patch to address the DBException handling that was brought up in the MAPREDUCE-5652 review and applies here. Note that this now depends upon YARN-1987 as that provides the utility wrapper for the leveldb iterator to translate raw RuntimeException to the more helpful DBException so we can act accordingly when errors occur. The other notable change in the patch is renaming LevelDB to Leveldb for consistency with the existing LeveldbTimelineStore naming convention. This latest patch includes the necessary pieces of YARN-1987 so it can compile and Jenkins can comment. Recover DeletionService state upon nodemanager restart -- Key: YARN-1339 URL: https://issues.apache.org/jira/browse/YARN-1339 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Affects Versions: 2.3.0 Reporter: Jason Lowe Assignee: Jason Lowe Attachments: YARN-1339.patch, YARN-1339v2.patch, YARN-1339v3-and-YARN-1987.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2009) Priority support for preemption in ProportionalCapacityPreemptionPolicy
[ https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986679#comment-13986679 ] Sunil G commented on YARN-2009: --- As per YARN-1963, jobs can be submitted in a queue with priority. In that scenario, when a queue is identified to preempt few containers among its applications, job priority also can be considered. Priority support for preemption in ProportionalCapacityPreemptionPolicy --- Key: YARN-2009 URL: https://issues.apache.org/jira/browse/YARN-2009 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler Reporter: Devaraj K While preempting containers based on the queue ideal assignment, we may need to consider preempting the low priority application containers first. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues
[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Shankar updated YARN-1864: - Attachment: YARN-1864-v4.txt Fair Scheduler Dynamic Hierarchical User Queues --- Key: YARN-1864 URL: https://issues.apache.org/jira/browse/YARN-1864 Project: Hadoop YARN Issue Type: New Feature Components: scheduler Reporter: Ashwin Shankar Labels: scheduler Attachments: YARN-1864-v1.txt, YARN-1864-v2.txt, YARN-1864-v3.txt, YARN-1864-v4.txt In Fair Scheduler, we want to be able to create user queues under any parent queue in the hierarchy. For eg. Say user1 submits a job to a parent queue called root.allUserQueues, we want be able to create a new queue called root.allUserQueues.user1 and run user1's job in it.Any further jobs submitted by this user to root.allUserQueues will be run in this newly created root.allUserQueues.user1. This is very similar to the 'user-as-default' feature in Fair Scheduler which creates user queues under root queue. But we want the ability to create user queues under ANY parent queue. Why do we want this ? 1. Preemption : these dynamically created user queues can preempt each other if its fair share is not met. So there is fairness among users. User queues can also preempt other non-user leaf queue as well if below fair share. 2. Allocation to user queues : we want all the user queries(adhoc) to consume only a fraction of resources in the shared cluster. By creating this feature,we could do that by giving a fair share to the parent user queue which is then redistributed to all the dynamically created user queues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues
[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987011#comment-13987011 ] Ashwin Shankar commented on YARN-1864: -- Hi [~sandyr] , bq. I think it's better to leave out case 4. The right behavior on it is fuzzy, and things are simpler if the results returned by QueuePlacementPolicy are only a function of the configuration. Removed queueMgr and using configuredQueues instead. bq. + MapFSQueueType,SetString configuredQueues; Need a space after the comma. Done bq. Both NestedUserQueue and UserQueueBelow sound good. Calling the rule 'NestedUserQueue'. Changed it at all places in the code. I didn't add documentation in my last patch, I've changed FS doc in the latest(v4) patch. Fair Scheduler Dynamic Hierarchical User Queues --- Key: YARN-1864 URL: https://issues.apache.org/jira/browse/YARN-1864 Project: Hadoop YARN Issue Type: New Feature Components: scheduler Reporter: Ashwin Shankar Labels: scheduler Attachments: YARN-1864-v1.txt, YARN-1864-v2.txt, YARN-1864-v3.txt, YARN-1864-v4.txt In Fair Scheduler, we want to be able to create user queues under any parent queue in the hierarchy. For eg. Say user1 submits a job to a parent queue called root.allUserQueues, we want be able to create a new queue called root.allUserQueues.user1 and run user1's job in it.Any further jobs submitted by this user to root.allUserQueues will be run in this newly created root.allUserQueues.user1. This is very similar to the 'user-as-default' feature in Fair Scheduler which creates user queues under root queue. But we want the ability to create user queues under ANY parent queue. Why do we want this ? 1. Preemption : these dynamically created user queues can preempt each other if its fair share is not met. So there is fairness among users. User queues can also preempt other non-user leaf queue as well if below fair share. 2. Allocation to user queues : we want all the user queries(adhoc) to consume only a fraction of resources in the shared cluster. By creating this feature,we could do that by giving a fair share to the parent user queue which is then redistributed to all the dynamically created user queues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987016#comment-13987016
]
Vinod Kumar Vavilapalli commented on YARN-1201:
---
That doesn't seem to make sense. When would localhost be not resolvable?
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-1945) Adding description for each pool in Fair Scheduler Page from fair-scheduler.xml
[ https://issues.apache.org/jira/browse/YARN-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated YARN-1945: -- Attachment: YARN-1945.v3.patch Adding description for each pool in Fair Scheduler Page from fair-scheduler.xml --- Key: YARN-1945 URL: https://issues.apache.org/jira/browse/YARN-1945 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.3.0 Reporter: Siqi Li Assignee: Siqi Li Attachments: YARN-1945.v2.patch, YARN-1945.v3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1354) Recover applications upon nodemanager restart
[ https://issues.apache.org/jira/browse/YARN-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1354: - Attachment: YARN-1354-v2-and-YARN-1987-and-YARN-1362.patch Updating the patch to address the DBException handling that was brought up in the MAPREDUCE-5652 review and applies here. Note that this now depends upon YARN-1987 as that provides the utility wrapper for the leveldb iterator to translate raw RuntimeException to the more helpful DBException so we can act accordingly when errors occur. This patch also addresses the issue where apps were being cleaned up on shutdown. This leverages YARN-1362 so we can distinguish a decommission shutdown, and it will avoid cleaning up applications if the state store can recover and we are not being decommissioned. The other notable change in the patch is renaming LevelDB to Leveldb for consistency with the existing LeveldbTimelineStore naming convention. This latest patch includes the necessary pieces of YARN-1987 and YARN-1362 so it can compile and Jenkins can comment. Recover applications upon nodemanager restart - Key: YARN-1354 URL: https://issues.apache.org/jira/browse/YARN-1354 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Affects Versions: 2.3.0 Reporter: Jason Lowe Assignee: Jason Lowe Attachments: YARN-1354-v1.patch, YARN-1354-v2-and-YARN-1987-and-YARN-1362.patch The set of active applications in the nodemanager context need to be recovered for work-preserving nodemanager restart -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1987) Wrapper for leveldb DBIterator to aid in handling database exceptions
[ https://issues.apache.org/jira/browse/YARN-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987059#comment-13987059 ] Ming Ma commented on YARN-1987: --- Jason, thanks for the clarification. The patch looks good to me. Wrapper for leveldb DBIterator to aid in handling database exceptions - Key: YARN-1987 URL: https://issues.apache.org/jira/browse/YARN-1987 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.4.0 Reporter: Jason Lowe Assignee: Jason Lowe Attachments: YARN-1987.patch Per discussions in YARN-1984 and MAPREDUCE-5652, it would be nice to have a utility wrapper around leveldb's DBIterator to translate the raw RuntimeExceptions it can throw into DBExceptions to make it easier to handle database errors while iterating. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1713) Implement getnewapplication and submitapp as part of RM web service
[
https://issues.apache.org/jira/browse/YARN-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987099#comment-13987099
]
shanyu zhao commented on YARN-1713:
---
[~vvasudev] I tested this patch and it works great!
I thought about your implementation of reflecting how RPC structures the API
and it sounds good to me.
The only feedback I have is the style of the REST API, does the following API
change make sense for you?
POST /app/id -- POST /appids
POST /app/{appid} -- POST /apps/{appid}
This way the submit app REST API is more consistent with app status and kill
app API.
Sorry I missed the part that your implementation actually calls doAs() on
behalf of the remote user. That should work just fine.
Thanks!
Implement getnewapplication and submitapp as part of RM web service
---
Key: YARN-1713
URL: https://issues.apache.org/jira/browse/YARN-1713
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-yarn-1713.3.patch, apache-yarn-1713.4.patch,
apache-yarn-1713.5.patch, apache-yarn-1713.cumulative.2.patch,
apache-yarn-1713.cumulative.3.patch, apache-yarn-1713.cumulative.4.patch,
apache-yarn-1713.cumulative.patch, apache-yarn-1713.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987102#comment-13987102
]
Wangda Tan commented on YARN-1201:
--
[~vinodkv], localhost cannot be resolved would be happened when it doesn't
contain a proper domain name, the mechanism used by Java to resolve a host is
similar to nslookup hostname. Domain name not configured is a common case
when user try Hadoop in local(vm).
As [~djp] suggested, I think we'd better throw a meaningful message rather than
NPE when hostname cannot be resolved.
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Moved] (YARN-2014) Performance: AM scaleability is 10% slower in 2.4 compared to 0.23.9
[ https://issues.apache.org/jira/browse/YARN-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli moved HADOOP-10553 to YARN-2014: Affects Version/s: (was: 2.4.0) 2.4.0 Key: YARN-2014 (was: HADOOP-10553) Project: Hadoop YARN (was: Hadoop Common) Performance: AM scaleability is 10% slower in 2.4 compared to 0.23.9 Key: YARN-2014 URL: https://issues.apache.org/jira/browse/YARN-2014 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: patrick white Performance comparison benchmarks from 2.x against 0.23 shows AM scalability benchmark's runtime is approximately 10% slower in 2.4.0. The trend is consistent across later releases in both lines, latest release numbers are: 2.4.0.0 runtime 255.6 seconds (avg 5 passes) 0.23.9.12 runtime 230.4 seconds (avg 5 passes) Diff: -9.9% AM Scalability test is essentially a sleep job that measures time to launch and complete a large number of mappers. The diff is consistent and has been reproduced in both a larger (350 node, 100,000 mappers) perf environment, as well as a small (10 node, 2,900 mappers) demo cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2014) Performance: AM scaleability is 10% slower in 2.4 compared to 0.23.9
[ https://issues.apache.org/jira/browse/YARN-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-2014: -- Target Version/s: 2.5.0 Tentatively setting target-version 2.5.0. Performance: AM scaleability is 10% slower in 2.4 compared to 0.23.9 Key: YARN-2014 URL: https://issues.apache.org/jira/browse/YARN-2014 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: patrick white Performance comparison benchmarks from 2.x against 0.23 shows AM scalability benchmark's runtime is approximately 10% slower in 2.4.0. The trend is consistent across later releases in both lines, latest release numbers are: 2.4.0.0 runtime 255.6 seconds (avg 5 passes) 0.23.9.12 runtime 230.4 seconds (avg 5 passes) Diff: -9.9% AM Scalability test is essentially a sleep job that measures time to launch and complete a large number of mappers. The diff is consistent and has been reproduced in both a larger (350 node, 100,000 mappers) perf environment, as well as a small (10 node, 2,900 mappers) demo cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1368) Common work to re-populate containers’ state into scheduler
[ https://issues.apache.org/jira/browse/YARN-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987155#comment-13987155 ] Vinod Kumar Vavilapalli commented on YARN-1368: --- bq. I think the prototype is doing as a scheduler-specifc fashion and only for the FairScheduler. It'll be a maintenance overhead if we implement each scheduler separately. This makes sense. As much as possible, RM restart should be orthogonal to schedulers. Further, where possible, we should try to reuse code within the schedulers. That way we won't miss stuff like metrics, scheduler calculations, resource consumption etc. Common work to re-populate containers’ state into scheduler --- Key: YARN-1368 URL: https://issues.apache.org/jira/browse/YARN-1368 Project: Hadoop YARN Issue Type: Sub-task Reporter: Bikas Saha Assignee: Anubhav Dhoot Attachments: YARN-1368.preliminary.patch YARN-1367 adds support for the NM to tell the RM about all currently running containers upon registration. The RM needs to send this information to the schedulers along with the NODE_ADDED_EVENT so that the schedulers can recover the current allocation state of the cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1368) Common work to re-populate containers’ state into scheduler
[ https://issues.apache.org/jira/browse/YARN-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987182#comment-13987182 ] Sandy Ryza commented on YARN-1368: -- +1 to sharing common code between schedulers where possible. However, the patch proposed on this JIRA leaves two separate accountings for the nodes in the cluster - one inside AbstractYarnScheduler and one inside the specific schedulers. As [~leftnoteasy] pointed out, if we want to merge scheduler code, it's probably best to do so in a patch that's separate from adding the re-population logic. A meta-comment - [~jianhe], if you have concerns about an approach / prototype it's better to provide feedback than to rewrite it yourself. Common work to re-populate containers’ state into scheduler --- Key: YARN-1368 URL: https://issues.apache.org/jira/browse/YARN-1368 Project: Hadoop YARN Issue Type: Sub-task Reporter: Bikas Saha Assignee: Anubhav Dhoot Attachments: YARN-1368.preliminary.patch YARN-1367 adds support for the NM to tell the RM about all currently running containers upon registration. The RM needs to send this information to the schedulers along with the NODE_ADDED_EVENT so that the schedulers can recover the current allocation state of the cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1368) Common work to re-populate containers’ state into scheduler
[ https://issues.apache.org/jira/browse/YARN-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987190#comment-13987190 ] Vinod Kumar Vavilapalli commented on YARN-1368: --- bq. Further, where possible, we should try to reuse code within the schedulers. To clarify, what I meant was not reuse across schedulers (which BTW is in itself a good goal), but reusing existing code that is inside schedulers to also do stuff when recovering from a restart. Anyways.. Common work to re-populate containers’ state into scheduler --- Key: YARN-1368 URL: https://issues.apache.org/jira/browse/YARN-1368 Project: Hadoop YARN Issue Type: Sub-task Reporter: Bikas Saha Assignee: Anubhav Dhoot Attachments: YARN-1368.preliminary.patch YARN-1367 adds support for the NM to tell the RM about all currently running containers upon registration. The RM needs to send this information to the schedulers along with the NODE_ADDED_EVENT so that the schedulers can recover the current allocation state of the cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2012) Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute
[ https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Shankar updated YARN-2012: - Attachment: YARN-2012-v1.txt Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute - Key: YARN-2012 URL: https://issues.apache.org/jira/browse/YARN-2012 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: Ashwin Shankar Labels: scheduler Attachments: YARN-2012-v1.txt Currently 'default' rule in queue placement policy,if applied,puts the app in root.default queue. It would be great if we can make 'default' rule optionally point to a different queue as default queue . This queue should be an existing queue,if not we fall back to root.default queue hence keeping this rule as terminal. This default queue can be a leaf queue or it can also be an parent queue if the 'default' rule is nested inside nestedUserQueue rule(YARN-1864). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2012) Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute
[ https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987205#comment-13987205 ] Ashwin Shankar commented on YARN-2012: -- Added an attribute 'queue' to default rule. Also added couple of unit tests. Patch is expected get -1 from Hadoop QA,since this jira depends upon YARN-1864. Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute - Key: YARN-2012 URL: https://issues.apache.org/jira/browse/YARN-2012 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: Ashwin Shankar Labels: scheduler Attachments: YARN-2012-v1.txt Currently 'default' rule in queue placement policy,if applied,puts the app in root.default queue. It would be great if we can make 'default' rule optionally point to a different queue as default queue . This queue should be an existing queue,if not we fall back to root.default queue hence keeping this rule as terminal. This default queue can be a leaf queue or it can also be an parent queue if the 'default' rule is nested inside nestedUserQueue rule(YARN-1864). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987208#comment-13987208
]
Wangda Tan commented on YARN-1201:
--
Uploaded a new patch to make this test case throw proper exception instead of a
simple NPE. This test cases hard coded checking
e.getCause().getMessage().contains(expectedMessage). When other exception
thrown, like UnknownHostException, the e.getCause().getMessage is null. So
updated type of exception cached from Exception to expected
AccessControlException.
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-1201:
-
Attachment: YARN-1201.patch
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch, YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1961) Fair scheduler preemption doesn't work for non-leaf queues
[
https://issues.apache.org/jira/browse/YARN-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987221#comment-13987221
]
Ashwin Shankar commented on YARN-1961:
--
hey [~sandyr], Do you know the history behind why preemption is implemented
only at a leaf queue level ?
Isn't this a bug ?
Just wanted to get some context.
Fair scheduler preemption doesn't work for non-leaf queues
--
Key: YARN-1961
URL: https://issues.apache.org/jira/browse/YARN-1961
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.4.0
Reporter: Ashwin Shankar
Labels: scheduler
Setting minResources and minSharePreemptionTimeout to a non-leaf queue
doesn't cause preemption to happen when that non-leaf queue is below
minResources and there are outstanding demands in that non-leaf queue.
Here is an example fs allocation config(partial) :
{code:xml}
queue name=abc
minResources3072 mb,0 vcores/minResources
minSharePreemptionTimeout30/minSharePreemptionTimeout
queue name=childabc1
/queue
queue name=childabc2
/queue
/queue
{code}
With the above configs,preemption doesn't seem to happen if queue abc is
below minShare and it has outstanding unsatisfied demands from apps in its
child queues. Ideally in such cases we would like preemption to kick off and
reclaim resources from other queues(not under queue abc).
Looking at the code it seems like preemption checks for starvation only at
the leaf queue level and not at the parent level.
{code:title=FairScheduler.java|borderStyle=solid}
boolean isStarvedForMinShare(FSLeafQueue sched)
boolean isStarvedForFairShare(FSLeafQueue sched)
{code}
This affects our use case where we have a parent queue with probably a 100
unconfigured leaf queues under it.We want to give a minshare to the parent
queue to protect all the leaf queues under it,but we cannot do it due to this
bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (YARN-2015) HTTPS doesn't work properly for daemons (RM, JHS, NM)
Robert Kanter created YARN-2015: --- Summary: HTTPS doesn't work properly for daemons (RM, JHS, NM) Key: YARN-2015 URL: https://issues.apache.org/jira/browse/YARN-2015 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Robert Kanter Assignee: Robert Kanter Priority: Blocker Enabling SSL in the site files and setting up a certificate, keystore, etc doesn't actually enable HTTPS. The RM, NMs, and JHS will use their https port, but use only HTTP on them instead of only HTTPS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2015) HTTPS doesn't work properly for daemons (RM, JHS, NM)
[ https://issues.apache.org/jira/browse/YARN-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987241#comment-13987241 ] Robert Kanter commented on YARN-2015: - The problem appears to be that while HttpServer2 is able to run an HTTPS endpoint and everything (there's even a test for it), the configs are not hooked up to actually enable it. HTTPS doesn't work properly for daemons (RM, JHS, NM) - Key: YARN-2015 URL: https://issues.apache.org/jira/browse/YARN-2015 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Robert Kanter Assignee: Robert Kanter Priority: Blocker Enabling SSL in the site files and setting up a certificate, keystore, etc doesn't actually enable HTTPS. The RM, NMs, and JHS will use their https port, but use only HTTP on them instead of only HTTPS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2015) HTTPS doesn't work properly for daemons (RM, JHS, NM)
[ https://issues.apache.org/jira/browse/YARN-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated YARN-2015: Affects Version/s: 2.3.0 HTTPS doesn't work properly for daemons (RM, JHS, NM) - Key: YARN-2015 URL: https://issues.apache.org/jira/browse/YARN-2015 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.3.0, 2.4.0 Reporter: Robert Kanter Assignee: Robert Kanter Priority: Blocker Enabling SSL in the site files and setting up a certificate, keystore, etc doesn't actually enable HTTPS. The RM, NMs, and JHS will use their https port, but use only HTTP on them instead of only HTTPS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (YARN-2015) HTTPS doesn't work properly for daemons (RM, JHS, NM)
[ https://issues.apache.org/jira/browse/YARN-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter resolved YARN-2015. - Resolution: Invalid Nevermind, this appears to be fixed by YARN-1553 HTTPS doesn't work properly for daemons (RM, JHS, NM) - Key: YARN-2015 URL: https://issues.apache.org/jira/browse/YARN-2015 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.3.0, 2.4.0 Reporter: Robert Kanter Assignee: Robert Kanter Priority: Blocker Enabling SSL in the site files and setting up a certificate, keystore, etc doesn't actually enable HTTPS. The RM, NMs, and JHS will use their https port, but use only HTTP on them instead of only HTTPS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1339) Recover DeletionService state upon nodemanager restart
[
https://issues.apache.org/jira/browse/YARN-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987261#comment-13987261
]
Hadoop QA commented on YARN-1339:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12642859/YARN-1339v3-and-YARN-1987.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3673//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3673//console
This message is automatically generated.
Recover DeletionService state upon nodemanager restart
--
Key: YARN-1339
URL: https://issues.apache.org/jira/browse/YARN-1339
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Affects Versions: 2.3.0
Reporter: Jason Lowe
Assignee: Jason Lowe
Attachments: YARN-1339.patch, YARN-1339v2.patch,
YARN-1339v3-and-YARN-1987.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987262#comment-13987262
]
Hadoop QA commented on YARN-1201:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12642965/YARN-1201.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3671//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3671//console
This message is automatically generated.
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch, YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1945) Adding description for each pool in Fair Scheduler Page from fair-scheduler.xml
[
https://issues.apache.org/jira/browse/YARN-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987264#comment-13987264
]
Hadoop QA commented on YARN-1945:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12642920/YARN-1945.v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1280 javac
compiler warnings (more than the trunk's current 1278 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3672//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/3672//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3672//console
This message is automatically generated.
Adding description for each pool in Fair Scheduler Page from
fair-scheduler.xml
---
Key: YARN-1945
URL: https://issues.apache.org/jira/browse/YARN-1945
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.3.0
Reporter: Siqi Li
Assignee: Siqi Li
Attachments: YARN-1945.v2.patch, YARN-1945.v3.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2012) Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute
[
https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987273#comment-13987273
]
Hadoop QA commented on YARN-2012:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12642962/YARN-2012-v1.txt
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3676//console
This message is automatically generated.
Fair Scheduler : Default rule in queue placement policy can take a queue as
an optional attribute
-
Key: YARN-2012
URL: https://issues.apache.org/jira/browse/YARN-2012
Project: Hadoop YARN
Issue Type: Improvement
Components: scheduler
Reporter: Ashwin Shankar
Labels: scheduler
Attachments: YARN-2012-v1.txt
Currently 'default' rule in queue placement policy,if applied,puts the app in
root.default queue. It would be great if we can make 'default' rule
optionally point to a different queue as default queue . This queue should be
an existing queue,if not we fall back to root.default queue hence keeping
this rule as terminal.
This default queue can be a leaf queue or it can also be an parent queue if
the 'default' rule is nested inside nestedUserQueue rule(YARN-1864).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-1201:
-
Attachment: YARN-1201.patch
Fixed incorrect type import
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch, YARN-1201.patch, YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1995) YARN REST APIs section contains both YARN and MapReduce REST API pages
[ https://issues.apache.org/jira/browse/YARN-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987329#comment-13987329 ] Akira AJISAKA commented on YARN-1995: - Thanks for the report [~zjshen]. Since the issue is also reported by YARN-1999 and there's more progress in the JIRA, would you please close this issue as duplicate? I appreciate you will review the patch in YARN-1999. YARN REST APIs section contains both YARN and MapReduce REST API pages Key: YARN-1995 URL: https://issues.apache.org/jira/browse/YARN-1995 Project: Hadoop YARN Issue Type: Bug Components: site Reporter: Zhijie Shen Priority: Minor Labels: newbie, site To be clear to users, it's good to either put the MapReduce related pages in a MAPREDUCE REST APIs section or rename current section name. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (YARN-1995) YARN REST APIs section contains both YARN and MapReduce REST API pages
[ https://issues.apache.org/jira/browse/YARN-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen resolved YARN-1995. --- Resolution: Duplicate [~ajisakaa], thanks for informing me of the duplication. Will look at the patch. YARN REST APIs section contains both YARN and MapReduce REST API pages Key: YARN-1995 URL: https://issues.apache.org/jira/browse/YARN-1995 Project: Hadoop YARN Issue Type: Bug Components: site Reporter: Zhijie Shen Priority: Minor Labels: newbie, site To be clear to users, it's good to either put the MapReduce related pages in a MAPREDUCE REST APIs section or rename current section name. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1201) TestAMAuthorization fails with local hostname cannot be resolved
[
https://issues.apache.org/jira/browse/YARN-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987357#comment-13987357
]
Hadoop QA commented on YARN-1201:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12642986/YARN-1201.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3677//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3677//console
This message is automatically generated.
TestAMAuthorization fails with local hostname cannot be resolved
Key: YARN-1201
URL: https://issues.apache.org/jira/browse/YARN-1201
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.1.0-beta
Environment: SUSE Linux Enterprise Server 11 (x86_64)
Reporter: Nemon Lou
Assignee: Wangda Tan
Priority: Minor
Attachments: YARN-1201.patch, YARN-1201.patch, YARN-1201.patch
When hostname is 158-1-131-10, TestAMAuthorization fails.
{code}
Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.034 sec
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
testUnauthorizedAccess[0](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.952 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
testUnauthorizedAccess[1](org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization)
Time elapsed: 3.116 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization.testUnauthorizedAccess(TestAMAuthorization.java:284)
Results :
Tests in error:
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
TestAMAuthorization.testUnauthorizedAccess:284 NullPointer
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
