[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[
https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756770#action_12756770
]
Aaron Kimball commented on MAPREDUCE-862:
-
You're right. I need more caffeine before using {{git blame}} in the future. :)
Moving discussion there.
> Modify UI to support a hierarchy of queues
> --
>
> Key: MAPREDUCE-862
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-862
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
>Reporter: Hemanth Yamijala
>Assignee: V.V.Chaitanya Krishna
> Fix For: 0.21.0
>
> Attachments: clustersummarymodification.png, detailspage.png,
> initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch,
> MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch,
> MAPREDUCE-862-6.patch, MAPREDUCE-862-7.patch, subqueue.png
>
>
> MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce
> framework. This JIRA is for defining changes to the UI related to queues.
> This includes the hadoop queue CLI and the web UI on the JobTracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756681#action_12756681 ] Hemanth Yamijala commented on MAPREDUCE-862: Also, it is unlikely this is caused by MAPREDUCE-862. If at all, it should be caused by MAPREDUCE-861. This patch did not touch the QueueManager code at all. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Fix For: 0.21.0 > > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, > MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch, > MAPREDUCE-862-6.patch, MAPREDUCE-862-7.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756668#action_12756668 ] Hemanth Yamijala commented on MAPREDUCE-862: Aaron, in Hadoop 0.21, we are shifting the queue configuration to mapred-queues.xml. And the system is expected to have atleast one queue - the default queue. This is ensured in a default setup, because there's a default mapred-queues.xml.template that's copied over to become mapred-queues.xml during build. However, for backwards compatibility, you could have queues defined in mapred-site.xml. In your case when conf/mapred-queues.xml did not exist, were there queues configured in mapred-site ? > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Fix For: 0.21.0 > > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, > MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch, > MAPREDUCE-862-6.patch, MAPREDUCE-862-7.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[
https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756663#action_12756663
]
Aaron Kimball commented on MAPREDUCE-862:
-
-1. My jobtracker now fails to start with a NullPointerException:
{code}
2009-09-17 10:45:46,121 INFO org.apache.hadoop.util.HostsFileReader: Refreshing
hosts (include/exclude) list
2009-09-17 10:45:46,124 FATAL org.apache.hadoop.mapred.JobTracker:
java.lang.NullPointerException
»- at
org.apache.hadoop.mapred.QueueManager.getQueueConfigurationParser(QueueManager.java:117)
»- at org.apache.hadoop.mapred.QueueManager.(QueueManager.java:137)
»- at org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1315)
»- at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:212)
»- at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:204)
»- at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:199)
»- at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3584)
2009-09-17 10:45:46,125 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG:
{code}
This happens if {{conf/mapred-queues.xml}} does not exist.
> Modify UI to support a hierarchy of queues
> --
>
> Key: MAPREDUCE-862
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-862
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
>Reporter: Hemanth Yamijala
>Assignee: V.V.Chaitanya Krishna
> Fix For: 0.21.0
>
> Attachments: clustersummarymodification.png, detailspage.png,
> initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch,
> MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch,
> MAPREDUCE-862-6.patch, MAPREDUCE-862-7.patch, subqueue.png
>
>
> MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce
> framework. This JIRA is for defining changes to the UI related to queues.
> This includes the hadoop queue CLI and the web UI on the JobTracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756648#action_12756648 ] Hudson commented on MAPREDUCE-862: -- Integrated in Hadoop-Mapreduce-trunk #85 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/85/]) . Enhance JobTracker UI to display hierarchical queues. Contributed by V.V.Chaitanya Krishna. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Fix For: 0.21.0 > > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, > MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch, > MAPREDUCE-862-6.patch, MAPREDUCE-862-7.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756502#action_12756502 ] Hudson commented on MAPREDUCE-862: -- Integrated in Hadoop-Mapreduce-trunk-Commit #45 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/45/]) . Enhance JobTracker UI to display hierarchical queues. Contributed by V.V.Chaitanya Krishna. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Fix For: 0.21.0 > > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, > MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch, > MAPREDUCE-862-6.patch, MAPREDUCE-862-7.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756421#action_12756421 ] Hadoop QA commented on MAPREDUCE-862: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12419845/MAPREDUCE-862-7.patch against trunk revision 816052. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/94/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/94/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/94/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/94/console This message is automatically generated. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Fix For: 0.21.0 > > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, > MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch, > MAPREDUCE-862-6.patch, MAPREDUCE-862-7.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756355#action_12756355 ] Hemanth Yamijala commented on MAPREDUCE-862: This is quite close. I have a few cleanup comments: 1. JobQueueClient.displayQueueAclsInfoForCurrentUser need not init JobConf again. 2. Are the changes in QueueManagerTestUtils required ? 3. CONFDIR and CONFFILEPATH in TestJobQueueClient seem unnecessary now. Please mark this patch available with the above changes. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, > MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch, > MAPREDUCE-862-6.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755999#action_12755999 ] Hemanth Yamijala commented on MAPREDUCE-862: Regarding the CLI changes, I think we must: - Minimize the changes to JobQueueClient and not make any changes to JobClient - because we aren't per se changing anything in the job client. - Convert the unit test to a proper unit test without needing a MiniMRCluster. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, > MAPREDUCE-862-3.patch, MAPREDUCE-862-4.patch, MAPREDUCE-862-5.patch, > subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[
https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755318#action_12755318
]
Sreekanth Ramakrishnan commented on MAPREDUCE-862:
--
The new files which were added for the UI are missing in the latest patch, can
you reupload the patch with those files included?
In jobtracker.jsp after removing of the scheduling information table, please
add a {{}} to terminate the section from Summary table.
> Modify UI to support a hierarchy of queues
> --
>
> Key: MAPREDUCE-862
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-862
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
>Reporter: Hemanth Yamijala
>Assignee: V.V.Chaitanya Krishna
> Attachments: clustersummarymodification.png, detailspage.png,
> initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, subqueue.png
>
>
> MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce
> framework. This JIRA is for defining changes to the UI related to queues.
> This includes the hadoop queue CLI and the web UI on the JobTracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755011#action_12755011 ] Hemanth Yamijala commented on MAPREDUCE-862: I looked at the CLI changes. Look mostly fine. Few minor comments: - displayQueueInfo: Redundant call to jobQueueInfo.getChildren can be removed. - Also, I think child queues should be printed before the jobs are shown as its more natural that way - Should we print the full name of the queue when displaying the queue name ? I think it will be convenient to do so. - Also, I think it will be useful to separate each queue info with a separator like "=" - I think we should try and write test cases using TestMRCli. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, MAPREDUCE-862-2.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754869#action_12754869 ] Sreekanth Ramakrishnan commented on MAPREDUCE-862: -- Took a look at patch: * Add link in cluster status table to redirecting to new queue information page. * Move the Queue information to a new JSP. redirect to this page if we are not able to connect to YUI servers. Hemanth's offline comment: * Add recursive option in CLI for queue -info * Add a new field in the queue -info which lists only top level child queues. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754098#action_12754098 ] V.V.Chaitanya Krishna commented on MAPREDUCE-862: - The uploaded patch contains the patch of MAPREDUCE-861 since it is required for the design. > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala >Assignee: V.V.Chaitanya Krishna > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, MAPREDUCE-862-1.patch, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-862) Modify UI to support a hierarchy of queues
[ https://issues.apache.org/jira/browse/MAPREDUCE-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750902#action_12750902 ] Hemanth Yamijala commented on MAPREDUCE-862: The mock-ups look fine to me. For the benefit of others, these were created using [YUI|http://developer.yahoo.com/yui/] that is available under [the BSD license|http://developer.yahoo.com/yui/license.html] which is an Apache compatible license. In a local discussion with folks, we decided that we could just let the YUI scripts be downloaded in a standard manner by the browser. This means that to view the queue related pages, we'd need a working internet connection. That seems a reasonable assumption to make, with a caveat that if there's no connection, it should fail gracefully. Other options to bundle the javascripts would be involve a lot of sync-up with YUI libraries, which was felt a hazzle to maintain. Thoughts ? > Modify UI to support a hierarchy of queues > -- > > Key: MAPREDUCE-862 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-862 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Hemanth Yamijala > Attachments: clustersummarymodification.png, detailspage.png, > initialscreen.png, subqueue.png > > > MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce > framework. This JIRA is for defining changes to the UI related to queues. > This includes the hadoop queue CLI and the web UI on the JobTracker. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
