[jira] [Updated] (YARN-576) RM should not allow registrations from NMs that do not satisfy minimum scheduler allocations

2013-04-28 Thread Kenji Kikushima (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kenji Kikushima updated YARN-576:
-

Attachment: YARN-576-4.patch

Sure. No problem.
Added private variables and moved reading conf to init. Also fixed typo.
Thanks.

 RM should not allow registrations from NMs that do not satisfy minimum 
 scheduler allocations
 

 Key: YARN-576
 URL: https://issues.apache.org/jira/browse/YARN-576
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Hitesh Shah
Assignee: Kenji Kikushima
  Labels: newbie
 Attachments: YARN-576-2.patch, YARN-576-3.patch, YARN-576-4.patch, 
 YARN-576.patch


 If the minimum resource allocation configured for the RM scheduler is 1 GB, 
 the RM should drop all NMs that register with a total capacity of less than 1 
 GB. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-576) RM should not allow registrations from NMs that do not satisfy minimum scheduler allocations

2013-04-28 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644038#comment-13644038
 ] 

Hadoop QA commented on YARN-576:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12580888/YARN-576-4.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/834//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/834//console

This message is automatically generated.

 RM should not allow registrations from NMs that do not satisfy minimum 
 scheduler allocations
 

 Key: YARN-576
 URL: https://issues.apache.org/jira/browse/YARN-576
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Hitesh Shah
Assignee: Kenji Kikushima
  Labels: newbie
 Attachments: YARN-576-2.patch, YARN-576-3.patch, YARN-576-4.patch, 
 YARN-576.patch


 If the minimum resource allocation configured for the RM scheduler is 1 GB, 
 the RM should drop all NMs that register with a total capacity of less than 1 
 GB. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (YARN-623) NodeManagers on RM web-app don't have diagnostic information

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)
Vinod Kumar Vavilapalli created YARN-623:


 Summary: NodeManagers on RM web-app don't have diagnostic 
information
 Key: YARN-623
 URL: https://issues.apache.org/jira/browse/YARN-623
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Vinod Kumar Vavilapalli


If RM for some reason asks NMs to shut-down or reboot, it will be very useful 
to show that information on the UI so that operators can know directly instead 
of login into machines and looking for logs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-576) RM should not allow registrations from NMs that do not satisfy minimum scheduler allocations

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644112#comment-13644112
 ] 

Vinod Kumar Vavilapalli commented on YARN-576:
--

+1, the latest patch looks good, checking it in.

In cases like these, it is useful to show such info on RM web app - filed 
YARN-623.

 RM should not allow registrations from NMs that do not satisfy minimum 
 scheduler allocations
 

 Key: YARN-576
 URL: https://issues.apache.org/jira/browse/YARN-576
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Hitesh Shah
Assignee: Kenji Kikushima
  Labels: newbie
 Attachments: YARN-576-2.patch, YARN-576-3.patch, YARN-576-4.patch, 
 YARN-576.patch


 If the minimum resource allocation configured for the RM scheduler is 1 GB, 
 the RM should drop all NMs that register with a total capacity of less than 1 
 GB. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-576) RM should not allow registrations from NMs that do not satisfy minimum scheduler allocations

2013-04-28 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644121#comment-13644121
 ] 

Hudson commented on YARN-576:
-

Integrated in Hadoop-trunk-Commit #3685 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/3685/])
YARN-576. Modified ResourceManager to reject NodeManagers that don't satisy 
minimum resource requirements. Contributed by Kenji Kikushima. (Revision 
1476824)

 Result = SUCCESS
vinodkv : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1476824
Files : 
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java


 RM should not allow registrations from NMs that do not satisfy minimum 
 scheduler allocations
 

 Key: YARN-576
 URL: https://issues.apache.org/jira/browse/YARN-576
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Hitesh Shah
Assignee: Kenji Kikushima
  Labels: newbie
 Fix For: 2.0.5-beta

 Attachments: YARN-576-2.patch, YARN-576-3.patch, YARN-576-4.patch, 
 YARN-576.patch


 If the minimum resource allocation configured for the RM scheduler is 1 GB, 
 the RM should drop all NMs that register with a total capacity of less than 1 
 GB. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-513) Verify all clients will wait for RM to restart

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644122#comment-13644122
 ] 

Vinod Kumar Vavilapalli commented on YARN-513:
--

See HDFS-3504 for related discussion.

 Verify all clients will wait for RM to restart
 --

 Key: YARN-513
 URL: https://issues.apache.org/jira/browse/YARN-513
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Reporter: Bikas Saha
Assignee: Xuan Gong
 Attachments: YARN-513.1.patch, YARN-513.2.patch


 When the RM is restarting, the NM, AM and Clients should wait for some time 
 for the RM to come back up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-606) negative queue metrics apps Failed

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644124#comment-13644124
 ] 

Vinod Kumar Vavilapalli commented on YARN-606:
--

Nemon, would you like to take this up?

 negative  queue metrics apps Failed
 -

 Key: YARN-606
 URL: https://issues.apache.org/jira/browse/YARN-606
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.0.3-alpha
Reporter: nemon lou
Priority: Minor

 Queue metrcis apps Failed can be negative in some cases(more than one 
 attempt for an application can cause this).
 It's confusing if we use this metrics directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (YARN-606) negative queue metrics apps Failed

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated YARN-606:
-

Target Version/s: 2.0.5-beta

 negative  queue metrics apps Failed
 -

 Key: YARN-606
 URL: https://issues.apache.org/jira/browse/YARN-606
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.0.3-alpha
Reporter: nemon lou
Priority: Minor

 Queue metrcis apps Failed can be negative in some cases(more than one 
 attempt for an application can cause this).
 It's confusing if we use this metrics directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Reopened] (YARN-516) TestContainerLocalizer.testContainerLocalizerMain is failing

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli reopened YARN-516:
--


Eli, you forgot to revert this from branch-2. Also should have reopened this 
for tracking purposes..

Reverting it myself and keeping it open for the sake of tracking YARN changes 
post HADOOP-9357.

 TestContainerLocalizer.testContainerLocalizerMain is failing
 

 Key: YARN-516
 URL: https://issues.apache.org/jira/browse/YARN-516
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Vinod Kumar Vavilapalli
Assignee: Andrew Wang
 Fix For: 2.0.5-beta

 Attachments: YARN-516.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (YARN-516) TestContainerLocalizer.testContainerLocalizerMain is failing

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated YARN-516:
-

Fix Version/s: (was: 2.0.5-beta)

 TestContainerLocalizer.testContainerLocalizerMain is failing
 

 Key: YARN-516
 URL: https://issues.apache.org/jira/browse/YARN-516
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Vinod Kumar Vavilapalli
Assignee: Andrew Wang
 Attachments: YARN-516.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Resolved] (YARN-620) TestContainerLocalizer.testContainerLocalizerMain failed on branch-2

2013-04-28 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli resolved YARN-620.
--

Resolution: Duplicate

This was caused by YARN-516, I reverted that patch from branch-2 also.

Jian, can you check once again if the test fails, just to be sure..? Tx.

 TestContainerLocalizer.testContainerLocalizerMain  failed on branch-2
 -

 Key: YARN-620
 URL: https://issues.apache.org/jira/browse/YARN-620
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
 Attachments: YARN-620.1.patch


 Argument(s) are different! Wanted:
 localFs.mkdir(
 
 /Users/jhe/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestContainerLocalizer/0/usercache/yak/filecache,
 isA(org.apache.hadoop.fs.permission.FsPermission),
 false
 );
 - at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestContainerLocalizer.testContainerLocalizerMain(TestContainerLocalizer.java:170)
 Actual invocation has different arguments:
 localFs.mkdir(
 
 file:/Users/jhe/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestContainerLocalizer/0/usercache/yak/filecache,
 rwxr-xr-x,
 false
 );
 - at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestContainerLocalizer.testContainerLocalizerMain(TestContainerLocalizer.java:162)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at 
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
   at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestContainerLocalizer.testContainerLocalizerMain(TestContainerLocalizer.java:170)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-513) Verify all clients will wait for RM to restart

2013-04-28 Thread Bikas Saha (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644174#comment-13644174
 ] 

Bikas Saha commented on YARN-513:
-

If this jira is going to address NM then how about we change the title to 
reflect that and open jiras for the other clients?

 Verify all clients will wait for RM to restart
 --

 Key: YARN-513
 URL: https://issues.apache.org/jira/browse/YARN-513
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Reporter: Bikas Saha
Assignee: Xuan Gong
 Attachments: YARN-513.1.patch, YARN-513.2.patch


 When the RM is restarting, the NM, AM and Clients should wait for some time 
 for the RM to come back up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-513) Verify all clients will wait for RM to restart

2013-04-28 Thread Bikas Saha (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644176#comment-13644176
 ] 

Bikas Saha commented on YARN-513:
-

Overall approach looks good. We could try to reuse existing RetryPolicy etc 
inside RMClient as long as we maintain the RMClient abstraction. Detailed 
review comments after we decide on what we want to resuse.
Are we not missing an RMClient.disconnect()? This one would internally stop the 
proxy?
Looks like NMStatusUpdater.getRMClient() can be removed because 
createRMClient() is being overridden by all tests.
Why are we throwing YARNException?
Is any test explicitly testing the new code with a real RM? How about manually 
doing it?

 Verify all clients will wait for RM to restart
 --

 Key: YARN-513
 URL: https://issues.apache.org/jira/browse/YARN-513
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Reporter: Bikas Saha
Assignee: Xuan Gong
 Attachments: YARN-513.1.patch, YARN-513.2.patch


 When the RM is restarting, the NM, AM and Clients should wait for some time 
 for the RM to come back up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-513) Verify all clients will wait for RM to restart

2013-04-28 Thread Xuan Gong (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644186#comment-13644186
 ] 

Xuan Gong commented on YARN-513:


After reading the code in HDFS-3504, we can definitely reuse the existing 
RetryPolicy code. We can use RetryPolicies.retryUpToMaximumTimeWithFixedSleep 
as defaultPolicy. We can also use RetryPolicies.retryByException to set a 
default policy with some explicit handlers for specific exceptions. 

 Verify all clients will wait for RM to restart
 --

 Key: YARN-513
 URL: https://issues.apache.org/jira/browse/YARN-513
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Reporter: Bikas Saha
Assignee: Xuan Gong
 Attachments: YARN-513.1.patch, YARN-513.2.patch


 When the RM is restarting, the NM, AM and Clients should wait for some time 
 for the RM to come back up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-578) NodeManager should use SecureIOUtils for serving logs and intermediate outputs

2013-04-28 Thread Omkar Vinit Joshi (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644199#comment-13644199
 ] 

Omkar Vinit Joshi commented on YARN-578:


Testing :- This patch I have tested on Secure setup.[ Ubuntu 12.04, single node 
using kerberos].

 NodeManager should use SecureIOUtils for serving logs and intermediate outputs
 --

 Key: YARN-578
 URL: https://issues.apache.org/jira/browse/YARN-578
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: nodemanager
Reporter: Vinod Kumar Vavilapalli
Assignee: Omkar Vinit Joshi
 Attachments: yarn-578-20130426.patch


 Log servlets for serving logs and the ShuffleService for serving intermediate 
 outputs both should use SecureIOUtils for avoiding symlink attacks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-326) Add multi-resource scheduling to the fair scheduler

2013-04-28 Thread Andrew Ferguson (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644208#comment-13644208
 ] 

Andrew Ferguson commented on YARN-326:
--

[~sandyr] bingo. that was exactly the concern I alluded to before. glad we 
found it while thinking about the design. :-)

[~kkambatl] yup, that's the idea -- fractional min-share, which would be 
interpreted as a fraction of the dominant resource (which wouldn't be 
pre-specified, so the queue's dominant resource could adapt based on the jobs 
submitted) ... I wrote my example a bit quickly, sorry! let me know if 
something's still not clear.

the new plan sounds like a good approach. I like it.



 Add multi-resource scheduling to the fair scheduler
 ---

 Key: YARN-326
 URL: https://issues.apache.org/jira/browse/YARN-326
 Project: Hadoop YARN
  Issue Type: New Feature
  Components: scheduler
Affects Versions: 2.0.2-alpha
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Attachments: FairSchedulerDRFDesignDoc.pdf, YARN-326.patch, 
 YARN-326.patch


 With YARN-2 in, the capacity scheduler has the ability to schedule based on 
 multiple resources, using dominant resource fairness.  The fair scheduler 
 should be able to do multiple resource scheduling as well, also using 
 dominant resource fairness.
 More details to come on how the corner cases with fair scheduler configs such 
 as min and max resources will be handled.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-326) Add multi-resource scheduling to the fair scheduler

2013-04-28 Thread Andrew Ferguson (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644209#comment-13644209
 ] 

Andrew Ferguson commented on YARN-326:
--

ps -- I forgot to include a pointer to the newest paper in the DRF line of 
work: http://www.cs.berkeley.edu/~matei/papers/2013/eurosys_choosy.pdf

 Add multi-resource scheduling to the fair scheduler
 ---

 Key: YARN-326
 URL: https://issues.apache.org/jira/browse/YARN-326
 Project: Hadoop YARN
  Issue Type: New Feature
  Components: scheduler
Affects Versions: 2.0.2-alpha
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Attachments: FairSchedulerDRFDesignDoc.pdf, YARN-326.patch, 
 YARN-326.patch


 With YARN-2 in, the capacity scheduler has the ability to schedule based on 
 multiple resources, using dominant resource fairness.  The fair scheduler 
 should be able to do multiple resource scheduling as well, also using 
 dominant resource fairness.
 More details to come on how the corner cases with fair scheduler configs such 
 as min and max resources will be handled.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-506) Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute

2013-04-28 Thread Ivan Mitic (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644215#comment-13644215
 ] 

Ivan Mitic commented on YARN-506:
-

Thanks Chris for the review!

bq. I still saw a failure in TestNodeHealthService after applying this patch. 
Ivan, did this patch fix that test for you?
Nope, the scope of the patch is to move to new common utils, and the test fails 
for a different reason. I validated that 
TestNodeHealthService#testNodeHealthScriptShouldRun succeeds with the patch 
(test revokes the execute permission on the script and validates that the 
script cannot run).

 Move to common utils FileUtil#setReadable/Writable/Executable and 
 FileUtil#canRead/Write/Execute
 

 Key: YARN-506
 URL: https://issues.apache.org/jira/browse/YARN-506
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic
 Attachments: YARN-506.commonfileutils.2.patch, 
 YARN-506.commonfileutils.patch


 Move to common utils described in HADOOP-9413 that work well cross-platform.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (YARN-506) Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute

2013-04-28 Thread Chris Nauroth (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644233#comment-13644233
 ] 

Chris Nauroth commented on YARN-506:


+1 for the patch.  Thank you, Ivan!

 Move to common utils FileUtil#setReadable/Writable/Executable and 
 FileUtil#canRead/Write/Execute
 

 Key: YARN-506
 URL: https://issues.apache.org/jira/browse/YARN-506
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic
 Attachments: YARN-506.commonfileutils.2.patch, 
 YARN-506.commonfileutils.patch


 Move to common utils described in HADOOP-9413 that work well cross-platform.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira