[jira] [Commented] (MAPREDUCE-1818) RaidNode should specify a pool name incase the cluster is using FairScheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016903#comment-13016903 ] Hudson commented on MAPREDUCE-1818: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > RaidNode should specify a pool name incase the cluster is using FairScheduler > - > > Key: MAPREDUCE-1818 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1818 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.20.1 >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali >Priority: Minor > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1818.2.patch, MAPREDUCE-1818.patch > > > contrib/fairscheduler (FairScheduler) supports scheduling based on pools. The > RaidNode should specify a pool name based on configuration to make use of > pools. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1818) RaidNode should specify a pool name incase the cluster is using FairScheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926153#action_12926153 ] Hudson commented on MAPREDUCE-1818: --- Integrated in Hadoop-Mapreduce-trunk-Commit #524 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/524/]) MAPREDUCE-1818. RaidNode can specify scheduling parameters. (Ramkumar Vadali via schen) > RaidNode should specify a pool name incase the cluster is using FairScheduler > - > > Key: MAPREDUCE-1818 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1818 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.20.1 >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali >Priority: Minor > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1818.2.patch, MAPREDUCE-1818.patch > > > contrib/fairscheduler (FairScheduler) supports scheduling based on pools. The > RaidNode should specify a pool name based on configuration to make use of > pools. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1818) RaidNode should specify a pool name incase the cluster is using FairScheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925620#action_12925620 ] Scott Chen commented on MAPREDUCE-1818: --- +1 Looks good to me. > RaidNode should specify a pool name incase the cluster is using FairScheduler > - > > Key: MAPREDUCE-1818 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1818 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.20.1 >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali >Priority: Minor > Attachments: MAPREDUCE-1818.2.patch, MAPREDUCE-1818.patch > > > contrib/fairscheduler (FairScheduler) supports scheduling based on pools. The > RaidNode should specify a pool name based on configuration to make use of > pools. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1818) RaidNode should specify a pool name incase the cluster is using FairScheduler
[
https://issues.apache.org/jira/browse/MAPREDUCE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925563#action_12925563
]
Ramkumar Vadali commented on MAPREDUCE-1818:
Test results:
ant test-patch
{code}
[exec]
[exec]
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 2 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
[exec]
[exec] +1 system tests framework. The patch passed system tests
framework compile.
[exec]
[exec]
[exec]
[exec]
[exec]
==
[exec]
==
[exec] Finished build.
[exec]
==
[exec]
==
[exec]
[exec]
BUILD SUCCESSFUL
Total time: 16 minutes 6 seconds
{code}
ant test under src/contrib/raid
{code}
test-junit:
[junit] WARNING: multiple versions of ant detected in path for junit
[junit]
jar:file:/home/rvadali/local/external/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
[junit] and
jar:file:/home/rvadali/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
[junit] Running org.apache.hadoop.hdfs.TestRaidDfs
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 47.02 sec
[junit] Running org.apache.hadoop.raid.TestDirectoryTraversal
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 8.871 sec
[junit] Running org.apache.hadoop.raid.TestErasureCodes
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 26.827 sec
[junit] Running org.apache.hadoop.raid.TestGaloisField
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.4 sec
[junit] Running org.apache.hadoop.raid.TestHarIndexParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.052 sec
[junit] Running org.apache.hadoop.raid.TestRaidHar
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 69.009 sec
[junit] Running org.apache.hadoop.raid.TestRaidNode
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 462.311 sec
[junit] Running org.apache.hadoop.raid.TestRaidPurge
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 44.044 sec
[junit] Running org.apache.hadoop.raid.TestRaidShell
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 21.899 sec
test:
BUILD SUCCESSFUL
Total time: 11 minutes 37 seconds
{code}
> RaidNode should specify a pool name incase the cluster is using FairScheduler
> -
>
> Key: MAPREDUCE-1818
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1818
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/raid
>Affects Versions: 0.20.1
>Reporter: Ramkumar Vadali
>Assignee: Ramkumar Vadali
>Priority: Minor
> Attachments: MAPREDUCE-1818.2.patch, MAPREDUCE-1818.patch
>
>
> contrib/fairscheduler (FairScheduler) supports scheduling based on pools. The
> RaidNode should specify a pool name based on configuration to make use of
> pools.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1818) RaidNode should specify a pool name incase the cluster is using FairScheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871970#action_12871970 ] Scott Chen commented on MAPREDUCE-1818: --- Capacity scheduler should have a configuration to specify which queue to use. I don't know the exact key. I think we should allow any kind of job configuration. And raid should not know about the details of the names and configuration keys. We can configure a parameter like raid.jobconf.location in each policy in raid.xml This raid.jobconf.location specify a path of a conf file for the raid job. For each policy, we can load the corresponding configuration file and passing them to the jobconf. This way we can support any parameters. And we don't have to create extra dependency between raid and other contrib projects. What do you think? > RaidNode should specify a pool name incase the cluster is using FairScheduler > - > > Key: MAPREDUCE-1818 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1818 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.20.1 >Reporter: Ramkumar Vadali >Priority: Minor > > contrib/fairscheduler (FairScheduler) supports scheduling based on pools. The > RaidNode should specify a pool name based on configuration to make use of > pools. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1818) RaidNode should specify a pool name incase the cluster is using FairScheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871944#action_12871944 ] Ramkumar Vadali commented on MAPREDUCE-1818: That is a good idea. Any pointers on what sort of configuration is useful for Capacity Scheduler? > RaidNode should specify a pool name incase the cluster is using FairScheduler > - > > Key: MAPREDUCE-1818 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1818 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.20.1 >Reporter: Ramkumar Vadali >Priority: Minor > > contrib/fairscheduler (FairScheduler) supports scheduling based on pools. The > RaidNode should specify a pool name based on configuration to make use of > pools. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1818) RaidNode should specify a pool name incase the cluster is using FairScheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871936#action_12871936 ] Scott Chen commented on MAPREDUCE-1818: --- It will be nice if we can make this more general. Raid should allow users to configure raid job in raid.xml. This way people use CapacityScheduler can also specify how to run the raid job. What do you think? > RaidNode should specify a pool name incase the cluster is using FairScheduler > - > > Key: MAPREDUCE-1818 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1818 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.20.1 >Reporter: Ramkumar Vadali >Priority: Minor > > contrib/fairscheduler (FairScheduler) supports scheduling based on pools. The > RaidNode should specify a pool name based on configuration to make use of > pools. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
