[jira] [Commented] (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016969#comment-13016969 ] Hudson commented on MAPREDUCE-2141: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Fix For: 0.22.0 > > Attachments: mapreduce-2141-v1.patch, mapreduce-2141-v2.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930907#action_12930907 ] Hudson commented on MAPREDUCE-2141: --- Integrated in Hadoop-Mapreduce-trunk-Commit #536 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/536/]) MAPREDUCE-2141. Add an "extra data" field to Task for use by Mesos. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch, mapreduce-2141-v2.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930373#action_12930373 ] Owen O'Malley commented on MAPREDUCE-2141: -- +1, looks good. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch, mapreduce-2141-v2.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924702#action_12924702 ] Matei Zaharia commented on MAPREDUCE-2141: -- The test failures seem to be unrelated to the patch. Let me know if there's anything else I need to do for it. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch, mapreduce-2141-v2.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924537#action_12924537 ] Hadoop QA commented on MAPREDUCE-2141: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12457646/mapreduce-2141-v2.patch against trunk revision 1026450. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system tests framework. The patch passed system tests framework compile. Test results: http://hudson.zones.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/3/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/3/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/3/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/3/console This message is automatically generated. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch, mapreduce-2141-v2.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922433#action_12922433 ] dhruba borthakur commented on MAPREDUCE-2141: - +1 on getting this into Apache Hadoop! > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922423#action_12922423 ] Tom White commented on MAPREDUCE-2141: -- Arun, I'm happy with this (using BytesWritable) too. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922420#action_12922420 ] Matei Zaharia commented on MAPREDUCE-2141: -- Thanks for the comments, Owen and Arun. We actually plan to submit Mesos to the Apache incubator soon, so maybe it will become Apache software eventually! I'll get back to you with an updated patch. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922416#action_12922416 ] Todd Lipcon commented on MAPREDUCE-2141: +1 for BytesWritable instead of String. Encoding is a pain in the butt. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922409#action_12922409 ] Owen O'Malley commented on MAPREDUCE-2141: -- I'd suggest using either BytesWritable or Text rather than String. Other than that it looks good. Some additional JavaDoc that explained that the field was for the scheduler would be helpful too, I think. > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Attachments: mapreduce-2141-v1.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
